sorry this is a very basic question. I wrote some functions to handle requests by passing a callback to http.createServer. The functions work if I import them directly from the functions.js file like in this code.
import {handle_request} from './functions.js';
import http from 'http';
http.createServer(handle_request).listen(8080);
Now I want to put this in a package, so that I can install the libraries my code depends on automatically with npm install. My package.json looks like this, I can install the package and I's found in the node REPL, but how do I actually call the functions in the package or use them as callbacks?
{
"name": "package_name",
"version": "1.0.0",
"description": "posts images to a slack channel",
"main": "functions.js",
"author": "me",
"dependencies": {
"mustache": "*"
},
"type": "module"
}
The functions are defined like this:
export var handle_request = function (req, res) {
// function body ...
}
I think my problem is either the import statements in the script that calls the library or some missing exports in the package.
Sorry again for the very basic question, I found many posts and documentations on npm packages but nothing solving my problem. Any help appreciated.
Posting the solution for sake of completeness. There were two mistakes I made:
there's a difference between import mechanisms inside and outside of a module context. The script calling the functions is not inside the module, therefore a using dynamic import is more appropriate.
dynamic import returns a promise, which I missed on an earlier try. In my case this is only called before the server starts, so I can just wait for the promise to be evaluated.
What worked in the end was:
let package_name = await import("package_name");
Thanks for the comment with the link to mdn, this helped a lot.
Related
I've written a library published to a private npm repo which is used by my applications.
This library contains utilities and has dependencies to other libraries, as an example let's choose #aws-sdk/client-lambda.
Some of my applications use only some of the utilities and don't need the dependencies to the external libraries, while some applications use all of the utilities.
To avoid having all applications getting a lot of indirect dependencies they don't need, I tried declaring the dependencies as peerDependencies and having the applications resolve the ones they need. It works well to publish the package, and to use it from applications who declare all of the peerDependencies as their own local dependencies, but applications failing to declare one of the dependencies get build errors when the included .d.ts files of the library are imported in application code:
error TS2307: Cannot find module '#aws-sdk/client-kms' or its corresponding type declarations.
Is it possible to resolve this situation so that my library can contain many different utils but the applications may "cherry-pick" the dependencies they need to fulfill the requirements of those utilities in runtime?
Do I have to use dynamic imports to do this or is there another way?
I tried using #ts-ignore in the library code, and it was propagated to the d.ts file imported by the applications, but it did not help.
Setup:
my-library
package.json:
peerDependencies: {
"#aws-sdk/client-lambda": "^3.27.0"
}
foo.ts:
import {Lambda} from '#aws-sdk/client-lambda';
export function foo(lambda: Lambda): void {
...
}
bar.ts:
export function bar(): void {
...
}
index.ts:
export * from './foo';
export * from './bar';
my-application1 - works fine
package.json:
dependencies: {
"my-library": "1.0.0",
"#aws-sdk/client-lambda": "^3.27.0"
}
test.ts:
import {foo} from 'my-library';
foo();
my-application2 - does not compile
package.json:
dependencies: {
"my-library": ...
}
test:ts:
import {bar} from 'my-library';
bar();
I found two ways of dealing with this:
1. Only use dynamic imports for the optional dependencies
If you make sure that types exported by the root file of the package only include types and interfaces and not classes etc, the transpiled JS will not contain any require statement to the optional library. Then use dynamic imports to import the optional library from a function so that they are required only when the client explicitly uses those parts of the library.
In the case of #aws-sdk/client-lambda, which was one of my optional dependencies, I wanted to expose function that could take an instance of a Lambda object or create one itself:
import {Lambda} from '#aws-sdk/client-lambda';
export function foo(options: {lambda?: Lambda}) {
if (!lambda) {
lambda = new Lambda({ ... });
}
...
}
Since Lambda is a class, it will be part of the transpiled JS as a require statement, so this does not work as an optional dependency. So I had to 1) make that import dynamic and 2) define an interface to be used in place of Lambda in my function's arguments to get rid of the require statement on the package's root path. Unfortunately in this particular case, the AWS SDK does not offer any type or interface which the class implements, so I had to come up with a minimal type such as
export interface AwsClient {
config: {
apiVersion: string;
}
}
... but of course, lacking a type ot represent the Lambda class, you might even resort to any.
Then comes the dynamic import part:
export async function foo(options: {lambda?: AwsClient}) {
if (!lambda) {
const {Lambda} = await import('#aws-sdk/client-lambda');
lambda = new Lambda({ ... });
}
...
}
With this code, there is no longer any require('#aws-sdk/client-lambda') on the root path of the package, only within the foo function. Only clients calling the foo function will have to have the dependency in their
node_modules.
As you can see, a side-effect of this is that every function using the optional library must be async since dynamic imports return promises. In my case this worked out, but it may complicate things. In one case I had a non-async function (such as a class constructor) needing an optional library, so I had no choice but to cache the promised import and resolve it later when used from an async member function, or do a lazy import when needed. This has the potential of cluttering code badly ...
So, to summarize:
Make sure any code that imports code from the optional library is put inside functions that the client wanting to use that functionality calls
It's OK to have imports of types from the optional library in the root of your package as it's stripped out when transpiled
If needed, defined substitute types to act as place-holders for any class arguments (as classes are both types and code!)
Transpile and investigate the resulting JS to see if you have any require statement for the optional library in the root, if so, you've missed something.
Note that if using webpack etc, using dynamic imports can be tricky as well. If the import paths are constants, it usually works, but building the path dynamically (await import('#aws-sdk/' + clientName)) will usually not unless you give webpack hints. This had me puzzled for a while since I wrote a wrapper in front of my optional AWS dependencies, which ended up not working at all for this reason.
2. Put the files using the optional dependencies in .ts files not exported by the root file of the package (i.e., index.ts).
This means that clients wanting to use the optional functionality must
import those files by sub-path, such as:
import {OptionalStuff} from 'my-library/dist/optional;
... which is obviously less than ideal.
in my case, the typescript IDE in vscode fails to import the optional type, so im using the relative import path
// fix: Cannot find module 'windows-process-tree' or its corresponding type declarations
//import type * as WindowsProcessTree from 'windows-process-tree';
import type * as WindowsProcessTree from '../../../../../node_modules/#types/windows-process-tree';
// global variable
let windowsProcessTree: typeof WindowsProcessTree;
if (true) { // some condition
windowsProcessTree = await import('windows-process-tree');
windowsProcessTree.getProcessTree(rootProcessId, tree => {
// ...
});
}
package.json
{
"devDependencies": {
"#types/windows-process-tree": "^0.2.0",
},
"optionalDependencies": {
"windows-process-tree": "^0.3.4"
}
}
based on vscode/src/vs/platform/terminal/node/windowsShellHelper.ts
trying to use passport-custom and the very first line, from pseudocode at npmjs, errors out:
import passportCustom from 'passport-custom';
The is no default import in index.js when I open it up under node_modules/passport-custom/lib
I must be missing something fundamental here, don't know what though
Try to use CommonJS const passportCustom = require("passport-custom") You probably have older version of Node.js which does not support ES6 modules.
There is no default export. So you will have to name the items you want to import (put them in curly braces).
//Example:
import { a,b,c,d} from 'youPackage';
//Your case:
import { passportCustom } from 'passport-custom';
Above are called named imports. When a package exports one item by default using: export default passportCustom ;, you could have use your code. You can access the code of the package to have a look for yourself.
I have an aws amplify project which is using yarn workspaces.
My project has a lambda function and a layer. When the lambda function runs in aws it needs to import my library from the layer...
import MyLib from '/opt/nodejs/build/MyLib';
However when I run this function locally I want to import the library from my local file system
import MyLib from '/Users/sive/Documents/SWT/wake-book/amplify/backend/function/wakebookLayer/lib/nodejs/build/MyLib';
How can I tell typescript to compile an import from /opt/nodejs/build/MyLib to the location on my local file system.
I tried using path mapping in my tsconfig.json
"baseUrl": ".",
"paths": {
"/opt/nodejs/build/MyLib": ["./amplify/backend/function/wakebookLayer/lib/nodejs/build/MyLib.d.ts"]
},
This stops vscode complaining about an unknown import, so it is being 'linked' correctly (I can cmd click my import /opt/nodejs/build/MyLib path and I am taken to the full file system path.)
But the compiled js is still using the wrong path.
The compiled js file looks like this
const MyLib_1 = __importDefault(require("/opt/nodejs/build/MyLib"));
I would expect it to remap the import path and to look like this
const MyLib_1 = __importDefault(require("/Users/sive/Documents/SWT/wake-book/amplify/backend/function/wakebookLayer/lib/nodejs/build/MyLib"));
If I manually edit the output js to have the full file system path it works (I can run my project locally).
Have I misunderstood how path mapping is supposed to work?
Is there a way to get my desired behaviour?
Updated edit: I don't think this has anything to do with the fact that the path itself is long or complex. I'm having trouble getting mine to pass through correctly as well. I think the issue has to do more with how you're (and I'm) implementing the path mapping itself.
To test this theory, you could move one of those dependencies to a very simple, short path and see if that has any impact on the behavior at all.
Edit: I'm sorry, I just realized that you were trying to do this earlier and the problem seems to be related to the path itself... I'm not sure this will be helpful at all, but I'm going to leave it here for reference for now. I'm quite happy to try to address this with you over comments/edits.
What you are looking for is called "Path Mapping". There are a few ways this might be handled depending on your project. If you're deploying via SAM, this article from AWS describes one possible solution: https://aws.amazon.com/blogs/compute/working-with-aws-lambda-and-lambda-layers-in-aws-sam/
The important excerpt from that is here:
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: sam app
Globals:
Function:
Timeout: 3
Runtime: nodejs8.10
Resources:
TempConversionFunction:
Type: AWS::Serverless::Function
Properties:
CodeUri: hello-world/
Handler: app.lambdaHandler
Events:
HelloWorld:
Type: Api
Properties:
Path: /{conversion}/{value}
Method: get
This change dropped out some comments and output parameters and
updated the function resource to TempConversionFunction. The primary
change is the Path: /{conversion}/{value} line. This enables you to
use path mapping for our conversion type and value.
I've also seen examples that leverage tsconfig, but this seems to be a separate solution. Again, here is a link to the documentation: https://www.typescriptlang.org/docs/handbook/module-resolution.html#path-mapping
And the relevant excerpt:
Sometimes modules are not directly located under baseUrl. For
instance, an import to a module "jquery" would be translated at
runtime to "node_modules/jquery/dist/jquery.slim.min.js". Loaders use
a mapping configuration to map module names to files at run-time, see
RequireJs documentation and SystemJS documentation.
The TypeScript compiler supports the declaration of such mappings
using paths property in tsconfig.json files. Here is an example for
how to specify the paths property for jquery.
{
"compilerOptions": {
"baseUrl": ".", // This must be specified if "paths" is.
"paths": {
"jquery": ["node_modules/jquery/dist/jquery"] // This mapping is relative to "baseUrl"
}
}
}
Without knowing more about how you deploy your code, it's not possible for me to give more a more exact answer. But I hope this helps.
I should note, that your try/catch block does work, but it wouldn't be considered a best practice.
It's been a year and a half since this post was first answered, and there still doesn't seem to be a widely publicized solution to this issue.
So in case anyone stumbles upon this, here's what I came up with:
I have a JAVASCRIPT (.js, not a .ts file) in all my lambda function handler directories whose single responsibility is to import the layer, at run time, from the appropriate path (at either the '/opt/nodejs' path or the local path on my computer). It does so using a try/catch mechanism, which I know is a sort of anti-pattern, but it works well for this scenario.
try{
module.exports = require("/opt/nodejs/index")
} catch {
module.exports = require("../../layer/index"); // my local path
}
Then, in my handler code I import the layer with this code snippet:
import layer = require("../../layer"); // path to aforementioned file
This solution is leveraging an obscure typescript feature referred to as 'export = syntax'
https://www.typescriptlang.org/docs/handbook/modules.html#export--and-import--require
It only works if the file that imports the switching layer code is a javascript (.js) file. It doesn't work if it's a typescript (.ts) file. So you'll also have to configure your tsconfig.json file with "allowJs": true
I trying to convert a nodejs project to TypeScript and while mostly I did not faced really difficult obstacles during this process, the codebase has few gotchas like this, mostly in startup code:
function prepareConfiguration() {
let cloudConfigLoader = require('../utils/cloud-config');
return cloudConfigLoader.ensureForFreshConfig().then(function() {
//do some stuff
});
}
I may be need just an advice on which approach for refactoring this has less code changes to be made to make it work in TypeScript fashion.
In response to comments, more details:
That require loads the node module, not a JSON file. From that module the ensureForFreshConfig function contacts with a cloud service to load a list of values to rebuild a configuration state object.
Problem is that mdule was made in standard node approach of "module is isngleton object" and its independencies include auth component that will be ready only when the shown require call is made. I know it is not best a way to do so..
Typesript does not allow "mport that module later" except with dynamyc import which is problematic as mentiond in comment.
The "good" approach is to refactor that part of startup and make the ensureForFreshConfig and its dependency to initiate its ntenras on demand via cnstructors.. I just hoped ofr some soluiton to reduce things to be remade durng this transition to the TypeScript
import { cloudConfigLoader } from '../utils/cloud-config'
async function prepareConfiguration() {
await cloudConfigLoader.ensureForFreshConfig()
// do some stuff
// return some-stuff
}
The function is to be used as follows.
await prepareConfiguration()
So I was recently hacking on a large Typescript project (https://github.com/BabylonJS/Babylon.js) and I noticed that they don't ever have to import anything, they just use their namespace and the rest is (seemingly) magic.
It got me thinking that I would like to use something similar for myself, so I started a simple typescript project to try it out.
tsconfig.json
{
"compilerOptions": {
"baseUrl": "src",
"outFile": "server.js"
}
}
src/main.ts
module Test {
console.log('Main started')
const server:Server = new Server()
}
src/Server.ts
// import * as http from 'http'
module Test {
export class Server {
constructor() {
console.log('Server initialized')
}
}
}
If I build this Typescript project then I get output like the following:
// import * as http from 'http'
var Test;
(function (Test) {
var Server = /** #class */ (function () {
function Server() {
console.log('Server initialized');
}
return Server;
}());
Test.Server = Server;
})(Test || (Test = {}));
var Test;
(function (Test) {
console.log('Main started');
var server = new Test.Server();
})(Test || (Test = {}));
So far, so good. The trouble is that I want to take advantage of some external modules, in this case namely the http module, so I uncomment the import line above and now Typescript reports:
src/server/Server.ts(1,1): error TS6131: Cannot compile modules using option 'outFile' unless the '--module' flag is 'amd' or 'system'.
Node uses the commonjs module system, so obviously setting either of those flags isn't going to help me much. I have none the less tried them as well as various other combinations of flags, but to no avail. I noticed that BabylonJS doesn't really use external imports like that, opting instead to declare them as external and provide them globally at execution time as script tags. Is there maybe an analogue to that for Node?
You can't have these two things at the same time, namely
How can I avoid always having to import my own code in Typescript?
and
I want to take advantage of some external modules
You can avoid imports only by not using external modules, and the result will be one giant script file that can use external dependencies only as globals created by scripts loaded via script tag, as you already noticed.
The language does not allow you to use external modules when you do this. If you have an import of external module at the top level, your file becomes a module and there is no way it could use code from your other files without importing them. And having import of external module inside a namespace is not allowed AFAIK.
That said, I don't think your question - "How can I avoid always having to import my own code in Typescript?" - has a valid premise. CommonJS module system is the solution for preventing large projects from becoming unmaintainable mess. It does not matter if some part of a project is your own code or some external dependency - if it's a separate part with well-defined interface, it should be packaged and consumed as a module.
The solution that worked for me is this:
Server.ts
declare var http: any;
namespace Test {
export class Server {
constructor() {
console.log('Server initialized')
const server = http.createServer()
server.listen()
}
}
}
Then I simply provide http at runtime, for example by prepending a var http = require('http') to the output. Thanks to artem for a nudge in the right direction.