TypeScript AMD compilation and "barrel" modules - node.js

I'm trying to set up a Node.js + TypeScript project using Intern for testing. Everything works fine when I compile the project using "commonjs" (which I do for the normal build); and TypeScript is equally happy when compiling for "amd", which is required by Intern. However, when passing the tests with intern-client, it complains about a couple of things:
First, imports from "index.ts" files (so-called "barrel" modules) won't work. My setup is something like this (everything in the same directory):
// index.ts
export { x } from './x'
// x.ts
export function x() {}
// x.test.ts
import { x } from '.' // "Error: Failed to load module ..."
In fact, the generated JavaScript code (for x.test.ts) looks something like this:
define(["require", "exports", "."], function (...) { ... })
And I'm not sure that AMD knows how to handle the ".".
The second issue happens under the same circumstances (TypeScript compiles happily, but intern-client complains). In summary, I get an error when doing:
import jsdom = require('jsdom')
Which I need to transform to:
const jsdom = require('jsdom')
For Intern to be able to deal with it.
Here is the tsconfig.json file I use to compile the tests:
{
"compilerOptions": {
"target": "es6",
"module": "amd",
"moduleResolution": "node",
"sourceMap": true,
"rootDir": "src",
"outDir": "build/tests",
"noImplicitAny": true,
"suppressImplicitAnyIndexErrors": true
}
}
And here is my intern.js configuration file, in case it helps:
define({
suites: ['build/tests/**/*.test.js'],
excludeInstrumentation: true,
filterErrorStack: true
})
Edit (2017-05-03)
To help understand the issue, here is an excerpt of the directory tree of the project:
build
tests // The compiled tests will end up here
src
core
utils
x.ts
x.test.ts
// Other files, each containing a function that I would like to unit-test...
intern.js
package.json
tsconfig.json
...

Regarding the first issue, AMD's handling of an import like '.' is different than Node's. While both of them will map '.' to a package, Node uses a default module name of index.js, while AMD uses main.js. To get things working in an AMD loader, you'll need to first define a package for '.', and then tell the AMD loader what default module to use for that package. Given your project layout, you could configure Intern like this:
loaderOptions: {
map: {
// When a module in src/ references 'src/utils', redirect
// it to 'utils'
'src': {
'src/utils': 'utils'
}
},
packages: [
// Define a package 'utils' with files in 'src/utils' that defaults
// to the module index.js
{ name: 'utils', location: 'src/utils', main: 'index.js' }
]
}
Regarding the second issue, its not clear what the problem actually is. Import statements will be transpiled into define dependencies by TypeScript, so Intern should never be seeing them.

Related

TypeScript on AWS Lambda: to bundle imports (how?) or not to bundle? or: Runtime.ImportModuleError: Cannot find module '#aws-sdk/..."

I have the following lambda.ts code I'm trying to make running on an AWS Lambda:
import 'aws-sdk'
import { /* bunch of stuff... */ } from "#aws-sdk/client-cloudwatch-logs";
import {Context, APIGatewayProxyResult} from 'aws-lambda';
import {DateTime} from "luxon";
export const lambdaHandler = async (event: any, context: Context): Promise<APIGatewayProxyResult> => {
/* ... stuff ... */
}
which gets transpiled to:
"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
exports.lambdaHandler = void 0;
require("aws-sdk");
//import {CloudWatchClient} from "#aws-sdk/client-cloudwatch";
const client_cloudwatch_logs_1 = require("#aws-sdk/client-cloudwatch-logs");
const luxon_1 = require("luxon");
const lambdaHandler = async (event, context) => {
/* ... transpiled stuff ... */
}
When hitting the button, I'm getting this Response:
{
"errorType": "Runtime.ImportModuleError",
"errorMessage": "Error: Cannot find module '#aws-sdk/client-cloudwatch-logs'\nRequire stack:\n- /var/task/lambda.js\n- /var/runtime/index.mjs",
"trace": [
"Runtime.ImportModuleError: Error: Cannot find module '#aws-sdk/client-cloudwatch-logs'",
"Require stack:",
"- /var/task/lambda.js",
"- /var/runtime/index.mjs",
" at _loadUserApp (file:///var/runtime/index.mjs:951:17)",
" at async Object.UserFunction.js.module.exports.load (file:///var/runtime/index.mjs:976:21)",
" at async start (file:///var/runtime/index.mjs:1137:23)",
" at async file:///var/runtime/index.mjs:1143:1"
]
}
I played a lot with tsconfig.json, trying many things from Google / GitHub / SO / Rumors / Astrology / Numerology / Praying (monotheistic, pantheon-dwelling, neither..), but it only made me more confused.
For instance:
Using the tsconfig.json from Building Lambda functions with TypeScript still emits a single transpiled .js file without embedding the imported #aws-sdk/client-cloudwatch-logs module in the emitted output lambda.js file
Installing the AWS Common Runtime (CRT) Dependency
states that I need to npm install #aws-sdk/... (naturally), but doesn't explain anything beyond, which makes me think that maybe I shouldn't bundle them at all, but simply import them (in the assumption that they are pre-defined/loaded in AWS's Lambda's runtime)
Runtime is Node.js 16.x, Handler is lambda.lambdaHandler (emitted file is called lambda.js), and this is my current tsconfig.json:
{
"$schema": "https://json.schemastore.org/tsconfig",
"compilerOptions": {
"module": "commonjs",
"moduleResolution": "Node",
"target": "ES2022",
"sourceMap": true,
"lib": [
"ES2021"
],
"typeRoots": ["node_modules/#types"],
"outDir": "build",
"baseUrl": "src",
"strict": true,
"forceConsistentCasingInFileNames": true,
"esModuleInterop": true,
"resolveJsonModule": true,
"inlineSources": true,
"rootDir": "src",
"preserveConstEnums": true,
"isolatedModules": true,
"incremental": true,
"importHelpers": true
},
"exclude": [
"node_modules",
"**/*.test.ts"
]
}
So I'm trying to understand:
Do I even need to bundle those imported modules (such as #aws-sdk/client-cloudwatch-logs) at all, or they are already loaded by AWS Lambda's runtime?
If I do need to bundle, then how? do I need to use some bundler or is it just a matter of configuring tsconfig.json properly?
If bundler isn't mandatory, then how do I setup tsconfig.json to emit those 3rd-party modules?
If a bundler is mandatory, then can they all fit (WebPack, Babel, etc..)? or since no frontend (index.html) is involved, then not all of them can fit?
AWS SDK for JavaScript v3 (AKA modular) is not installed globally in the lambda execution context. You are using a v3 module (#aws-sdk/client-cloudwatch-logs) which is why it fails. AWS SDK v2 is installed globally, you are also using it (aws-sdk) so that require works fine.
You should use a bundler like webpack, esbuild, parcel or rollup. If you are using AWS CDK, there is a nodejs function construct that will do the bundling with esbuild for you.
TS will only emit your compiled javascript. If you are depending on javascript found in your node_modules directory, simply include that directory in your deployment package.
Generally, bundlers will take your application entry points (main.js, handler.js, whatever you want really) and recursively resolve all the dependencies, tree-shake any unreachable code then create one file for each entry point that has no other external dependencies. There is a runtime performance cost to this of course but it does simplify things and in a serverless context it isn't usually too impactful.
So, to resolve your error you can take one of two approaches:
Include your node_modules directory in your deployment package. (trivial)
Use a bundler or CDK (more complex)
Note that in either case, you need to be careful about dependencies with native bindings (binaries basically) as the one installed on your dev machine likely isn't supported in the lambda environment.

Unit test for aws lambda using jest

const invokeApi = require("/opt/nodejs/kiwiCall");
const decrypt = require("/opt/nodejs/encryption");
const cors = require("/opt/nodejs/cors");
When I am testing my index.js file by manual mocking these dependencies in mocks directory as follows:
__mocks__
|_invokeApi
|_decrypt
|_cors
it says
FAIL ./index.test.js
● Test suite failed to run
Cannot find module '/opt/nodejs/kiwiCall' from 'index.js'
However, Jest was able to find:
'../../../../lambdas/Flights/Locations/index.js'
You might want to include a file extension in your import, or update your 'moduleFileExtensions', which is currently ['js', 'json', 'jsx', 'ts', 'tsx', 'node'].
See https://jestjs.io/docs/en/configuration#modulefileextensions-array-string
1 | "use strict";
2 |
> 3 | const invokeApi = require("/opt/nodejs/kiwiCall");
Wanted to know how can I mock the dependencies of AWS lambda in inedx.test.js file
In your package.json or jest.config you could add a moduleNameMapper for that directory.
"jest": {
"moduleNameMapper": {
"/opt/nodejs/(.*)": "<rootDir>/../nodejs/$1"
},
},
So I managed to figure out something based on my repository.
I'm using the moduleNameMapper to map the absolute path to another location in my repository to where I have the layer stored.
Eg.
moduleNameMapper: {'^/opt/config/config': '<rootDir>/src/layers/layers-core/config/config'}
In your case you could use a regex expression to match /opt/nodejs/ and map it elsewhere. Hope that helped.
EDIT:
I completely changed my approach and used babel-plugin-module-resolver with babel-rewire. I did this because the above method was incompatible with rewire. It's quite easy setup and you just need to setup a babel alias within .babelrc.
eg.
{
"plugins": [
["rewire"],
["babel-plugin-module-resolver", {
"alias": {
"/opt/config/config": "./src/layers/layers-core/config/config",
"/opt/utils/util-logger": "./src/layers/layers-core/utils/util-logger",
"/opt/slack": "./src/layers/layers-slack/slack"
}
}]
]
}
Combine this with IDE jsconfig.json path alias and you get full IDE support.
{
"compilerOptions": {
"module": "commonjs",
"target": "es2018",
"baseUrl": "./",
"paths": {
"/opt/config/config": ["src/layers/layers-core/config/config"],
"/opt/utils/util-logger": ["src/layers/layers-core/utils/util-logger"],
"/opt/slack/*": ["src/layers/layers-slack/slack/*"],
}
},
"exclude": ["node_modules", "dist"]
}
You can then reference your layers with jest.doMock('/opt/config/config', mockConfig);
EDIT 2:
Found a way to get Jest to mock it. Just slip {virtual: true} into the mock!
jest.doMock('/opt/config/config', mockConfig, {virtual: true});
I have pretty much the same issue. I have defined a layer which contains common code that's shared between other functions in my project. My project structure looks something like this:
project/
functions/
function1/
app.js
function2/
app.js
shared/
shared.js
I import my shared library like this:
const { doSomething } = require('/opt/shared');
exports.handler = async (event) => {
const result = await doSomething();
// etc...
return {statusCode: 200};
}
This works when I deploy to AWS Lambda because the /opt/shared exists and it can be referenced correctly. It also works if I run this on my machine using sam local invoke Function1 because it's running in a container, which makes /opt/shared available to the code.
However, I'm struggling to work out how I can mock this dependency in a unit test. If I simply do this: jest.mock('/opt/shared'), I'm getting: Cannot find module '/opt/shared' from app.test.js
You can use the modulePaths option, from this post.
Documentation
jest.config.js
"jest": {
"modulePaths": [
"<rootDir>/src/layers/base/nodejs/node_modules/"
]
}
You can dynamically create this array by scanning a directory
const testFolder = './functions/';
const fs = require('fs');
const modulePaths = fs.readdirSync(testFolder)
.reduce((modulePaths, dirName) => {
modulePaths.push(`functions/${dirName}/dependencies/nodejs/node_modules/`);
return modulePaths;
}, []);

exports is not defined when running compiled typescript

I am trying to take my first steps into working with typescript and I've run into an issue when trying to run my application.
I get the error ReferenceError: exports is not defined
the code I have is quite simple:
// --src/changeset.ts
export enum ChangeAction {
ADD,
DELETE,
MODIFY
}
export class Changeset {
constructor(
public version: Number,
public content: String,
public path: String,
public action: ChangeAction
) {}
}
// --src/index.ts
import { Changeset, ChangeAction } from "./changeset";
const set = new Changeset(0, "Hello world", "/dev/null", ChangeAction.ADD);
set.version = 0;
console.log("Hello World! " + set.version);
// --tsconfig.json
{
"compilerOptions": {
"target": "es5",
"module": "commonjs",
"outDir": "build"
},
"include": ["src/**/*"]
}
running tsc, it compiles and seems to work without any real issues, however when I try to run it with node build/index.js it crashes with this
build/index.js:2
Object.defineProperty(exports, "__esModule", { value: true });
^
ReferenceError: exports is not defined
It feels like I am missing something quite obvious, but I can't really seem to put my finger on it, so what am I missing?
You appear to have enabled Node's ES modules by setting "type": "module" in your package.json, but your tsconfig tells typescript to emit code compatible with CommonJS.
Either remove "type": "module", or configure tsconfig to emit code targeting ES modules.

ReferenceError: Node is not defined (trying to use Node interface in typescript function in nodejs application)

While extending Cheerio library, I implemented the following static function (other extension functions work fine):
$.nodeType = function (elem: CheerioElement): number {
switch (elem.type) {
case "comment":
return Node.COMMENT_NODE; // <--- it fails here
case "tag":
return Node.ELEMENT_NODE; // <--- it fails here
case "text":
return Node.TEXT_NODE; // <--- it fails here
default:
return -1;
}
};
The following error appears during runtime (compilation with tsc -b succeed):
ReferenceError: Node is not defined
Node interface is part of the DOM API. Thus, I realized the need of explicitly include the DOM API under compilerOptions section of tsconfig.json.
However, I still get that runtime error.
Minimal relevant part of tsconfig.json:
{
"compilerOptions": {
"baseUrl": ".",
"incremental": true,
"lib": [
"esnext",
"dom"
],
"module": "commonjs",
"noImplicitAny": true,
"outDir": "./lib/",
"sourceMap": true,
"target": "esnext",
"watch": true
},
"include": [
"./src/**/*.ts",
]
}
I thought of explicitly import Node lib in the specific .ts file which contains the function, but I didn't find any "include-able" standard DOM lib.
Including a typescript lib doesn't polyfill the feature in an environment where it is not available.
While including "dom" type definitions will make the types available (at the compile time), but it doesn't actually make the Node API (typically provided by the browser runtime) available at runtime.
If you really need this functionality at runtime, you will need to also include an implementation of DOM for node.js such as jsdom which provides this API.
lorefnon explained the problem; here's a slightly hacky way to fix it:
Install jsdom
npm install jsdom
Add this to the top of your file:
const { JSDOM } = require('jsdom'); // or import { JSDOM } from 'jsdom';
const Node = new JSDOM('').window.Node;

Importing typescript from external node modules

I want to split my application into different node modules and have a main module which builds all other modules as well and I want to use typescript with es6 modules.
Here is my planned project structure:
main
node_modules
dep-a
dep-b
framework
interfaces
IComponent.ts
dep-a
components
test.ts
node_modules
framework
index.ts
dep-b
node_modules
framework
I want to be able to define interfaces in framework which can be consumed in dep-a, dep-b and main.
How do I set up this correctly? Can I compile everything from my main-module? Do I need to create different bundles for framework, dep-a, ... and another typing file? What is the best approach for this?
I already set up some test files and folders and used npm link to link the dependencies and webpack to bundle the files and I am always running into issues with files not being found:
error TS2307: Cannot find module 'framework/interfaces/IComponent'
and
Module not found: Error: Cannot resolve 'file' or 'directory' ./components/test
TL;DR generate declarations for the modules using declaration: true in tsconfig.json and specify the file for your generated typings in the typings entry of the package.json file
framework
Use a tsconfig file similar to this:
{
"compilerOptions": {
"target": "es5",
"module": "commonjs",
"declaration": true,
"noImplicitAny": true,
"removeComments": true,
"outDir": "dist",
...
},
"files": [
...
]
}
The important bit is declaration: true which will generate internal declarations in the dist directory
Assuming there is an index.ts file which (re)exports all the interesting parts of framework, create a package.json file with a main and typings entry pointing to, respectively, the generated js and the generated declaration, i.e.
{
"name": "framework",
"main": "dist/index.js",
"typings": "dist/index.d.ts",
...
}
Commit this module to a git repo, say bitbucket at : "https://myUser#bitbucket.org/myUser/framework.git"
dep-a
in package.json create a dependency to framework
{
"dependencies": {
"framework": "https://myUser#bitbucket.org/myUser/framework.git"
},
}
That is it.
import * from 'framework'
will pull the dependency with the typings, automatically
Obviously, it is possible to do with dep-a what was done with framework i.e. generate the declarations, update package.json and use dep-a as a module with embedded typings in main
note: a file URL will do in package.json/dependencies if you do not want go to via an external git repo
What arrived in TypeScript 1.6 is typings property in package.json module. You can check the relevant issue on GitHub.
So assuming you want to create separate modules ( dep-a, framework ). You can do the following :
main.ts // (1)
package.json // (2)
node_modules/
dep_a/
index.js // (3)
index.d.ts // (4)
package.json // (5)
node_modules/
framework/
index.js // (6)
index.d.ts // (7)
package.json // (8)
So let's see what you have in your files :
//(1) main.ts
import * as depA from "depA";
console.log(depA({ a : true, b : 2 }) === true) // true;
//(2) package.json
{
name: "main",
dependencies: {
"dep_a" : "0.0.1"
}
...
}
For depA
//(3) dep_a/index.js
module.exports = function a(options) { return true; };
//(4) dep_a/index.d.ts;
import * as framework from "framework";
export interface IDepA extends framework.IFramework {
a : boolean
}
export default function a(options: IDepA) : boolean;
//(5) dep_a/package.json
{
name: "dep_a",
dependencies: {
"framework" : "0.0.1"
},
...
typings : "index.d.ts" // < Magic happens here
}
For framework
//(6) dep_a/node_modules/framework/index.js
module.exports = true // we need index.js here, but we will only use definition file
//(7) dep_a/node_modules/framework/index.d.ts;
export interface IFramework {
b : number;
}
//(8) dep_a/node_modules/framework/package.json
{
name: "framework"
...
typings : "index.d.ts"
}
What I don't include in this answer ( for clarity ) is another compilation phase, so you could actually write the modules ( dep_a, framework ) with typescript and then compile them to index.js before you use them.
For a detailed explanation and some background also see: https://medium.com/#mweststrate/how-to-create-strongly-typed-npm-modules-1e1bda23a7f4

Resources