How do you access the url query params in nodejs? - node.js

How do you access the url query params in nodejs with typescript?
What I tried is here?
/**
* My Server app
*/
import * as Http from "http";
import * as url from "url";
import { HttpServer } from "./HttpServer";
import { TaxCalculator } from "./TaxCalculator";
export interface myReq extends Http.IncomingMessage {
amount: string;
rate: string;
}
/**
* MyServer class
* Create MyServer from HttpServer
*/
class MyServer extends HttpServer {
/**
* Create a new NodeServer object
* #param port Port number of the server
* #param name Name of the server
*/
constructor(port: number, name: string) {
super(port, name);
}
/**
* Handle the Request in request nad populate the response to response
* #param request Incoming Request
* #param response Outgoing Response
*/
onRequest(request: myReq, response: Http.ServerResponse): void {
response.writeHead(200, {"Content-Type": "text/plain"});
// const { amount, rate } = url.parse(request.url, true).query;
const query = url.parse(request.url, true).query;
const tc = new TaxCalculator();
const tax = tc.calculate(query.amount, query.rate);
response.end(JSON.stringify(tax));
}
}
// Create a server instance
const port = 8080;
new MyServer(port, "Test server");
The error is Argument of type 'string | string[]' is not assignable to parameter of type 'n
umber'.

Inside your route handling middleware there are the req, res, next parameters of the router callback. You are looking for req.query.[your query param].
For typescript, you'll want to create an interface that extends express.Request and add your query param there. Then assign that type to your request param.
interface myReq extends express.Request {
query: {
[Whatever params you have in your route]: string;
}
}
router.get("/", (req: myReq, res, next) => {
...
})
Youll also have to import express in the file if you havent already

Related

Accessing middleware injected data in Typescript

Here's a high level view of a controller in Typescript-Node :
As I'm storing details of user in product model, I have used a middleware to check if user if logged in before accessing the endpoint and also injecting user info to the req which can be further used in different controllers
exports.addProduct = async (req: Request, res: Response, next: NextFunction) => {
// images:
try {
// logic to handle data from req.body
// getting this user id from middleware isLoggedIn
// injecting user id into request in the isLoggedIn middleware
req.body.user = req.user._id;
const product = await Product.create(req.body);
return res.status(200).json({
success: true,
product,
});
} catch (error) {
logger.error(error);
}
};
Getting error : Property 'user' does not exist on type 'Request<ParamsDictionary, any, any, ParsedQs, Record<string, any>>' , on the line
req.body.user = req.user._id;
isLoggedIn is typical function to check Bearer token or header or cookies and then inject user info to the request
It worked Perfectly in Javascript, now trying same in Typescript as a part to learn Typescipt
There are two ways to achieve this:
Extending express Request locally
Extending express Request globally
Using the local way require to write lots of redundent code and that's why
the global way is much better. it can be done by creating file as follows:
index.d.ts
import { User } from "../../models/user";
// to make the file a module and avoid the TypeScript error
export {};
declare global {
namespace Express {
export interface Request {
user: User;
}
}
}
Then add this config to tsconfig.json file
"typeRoots": [
"src/#types",
"./node_modules/#types",
],
Then Request object will recognize user and user can be injected from any middleware to be used in any controller.
The problem is that according to the typing of req, there is no property named user. TypeScript is notifying you that req.user should be undefined, according to the available typings. There are some possible solutions to fix your problem.
You could explicitly type the variable as any. This is considered to be bad practice sometimes, because in general you should try to type everything correctly (nevertheless: it works).
// Option 1: Explicitly declare variable as any
req.body.user = (req as any).user._id;
You could also check if req.user is defined, like this:
// Option 2: Check req.user manually
if (req.user) req.body.user = req.user._id;
else throw new Error("Some Error");
You could also type the req correctly, according to the API specifications of your middleware. This is usually a lot of work if done manually. Some modules ship with correct TypeScript-typings already.
Maybe you want to also look into this question since it is very similar to your question.
first create a folder call types it should be at the root of your project
then at yow tsconfig.json in the compilerOptions section add a paths prop
{
"compilerOptions": {
...
"paths": {
"express": [
"./types/express/index.d.ts"
],
}
}
then at the types dir add a new dir call express inside add an index.d.ts go ahead a copy them express definitions
// Type definitions for Express 4.17
// Project: http://expressjs.com
// Definitions by: Boris Yankov <https://github.com/borisyankov>
// China Medical University Hospital <https://github.com/CMUH>
// Puneet Arora <https://github.com/puneetar>
// Dylan Frankland <https://github.com/dfrankland>
// Definitions: https://github.com/DefinitelyTyped/DefinitelyTyped
/* =================== USAGE ===================
import express = require("express");
var app = express();
=============================================== */
/// <reference types="express-serve-static-core" />
/// <reference types="serve-static" />
import * as bodyParser from 'body-parser';
import * as serveStatic from 'serve-static';
import * as core from 'express-serve-static-core';
import * as qs from 'qs';
/**
* Creates an Express application. The express() function is a top-level function exported by the express module.
*/
declare function e (): core.Express;
declare namespace e {
/**
* This is a built-in middleware function in Express. It parses incoming requests with JSON payloads and is based on body-parser.
* #since 4.16.0
*/
var json: typeof bodyParser.json;
/**
* This is a built-in middleware function in Express. It parses incoming requests with Buffer payloads and is based on body-parser.
* #since 4.17.0
*/
var raw: typeof bodyParser.raw;
/**
* This is a built-in middleware function in Express. It parses incoming requests with text payloads and is based on body-parser.
* #since 4.17.0
*/
var text: typeof bodyParser.text;
/**
* These are the exposed prototypes.
*/
var application: Application;
var request: Request;
var response: Response;
/**
* This is a built-in middleware function in Express. It serves static files and is based on serve-static.
*/
var static: serveStatic.RequestHandlerConstructor<Response>;
/**
* This is a built-in middleware function in Express. It parses incoming requests with urlencoded payloads and is based on body-parser.
* #since 4.16.0
*/
var urlencoded: typeof bodyParser.urlencoded;
/**
* This is a built-in middleware function in Express. It parses incoming request query parameters.
*/
export function query (options: qs.IParseOptions | typeof qs.parse): Handler;
export function Router (options?: RouterOptions): core.Router;
interface RouterOptions {
/**
* Enable case sensitivity.
*/
caseSensitive?: boolean | undefined;
/**
* Preserve the req.params values from the parent router.
* If the parent and the child have conflicting param names, the child’s value take precedence.
*
* #default false
* #since 4.5.0
*/
mergeParams?: boolean | undefined;
/**
* Enable strict routing.
*/
strict?: boolean | undefined;
}
interface SessionData {
userIp: string;
ipDetails: any;
publicKey: string;
session: string;
iv: string;
decrypted: any;
}
interface Application extends core.Application { }
interface CookieOptions extends core.CookieOptions { }
interface Errback extends core.Errback { }
interface ErrorRequestHandler<
P = core.ParamsDictionary,
ResBody = any,
ReqBody = any,
ReqQuery = core.Query,
Locals extends Record<string, any> = Record<string, any>
> extends core.ErrorRequestHandler<P, ResBody, ReqBody, ReqQuery, Locals> { }
interface Express extends core.Express { }
interface Handler extends core.Handler { }
interface IRoute extends core.IRoute { }
interface IRouter extends core.IRouter { }
interface IRouterHandler<T> extends core.IRouterHandler<T> { }
interface IRouterMatcher<T> extends core.IRouterMatcher<T> { }
interface MediaType extends core.MediaType { }
interface NextFunction extends core.NextFunction { }
interface Request<
P = core.ParamsDictionary,
ResBody = any,
ReqBody = any,
ReqQuery = core.Query,
Locals extends Record<string, any> = Record<string, any>
> extends core.Request<P, ResBody, ReqBody, ReqQuery, Locals> { }
interface RequestHandler<
P = core.ParamsDictionary,
ResBody = any,
ReqBody = any,
ReqQuery = core.Query,
Locals extends Record<string, SessionData> = Record<string, SessionData>
> extends core.RequestHandler<P, ResBody, ReqBody, ReqQuery, Locals> { }
interface RequestParamHandler extends core.RequestParamHandler { }
export interface Response<ResBody = any, Locals extends Record<string, SessionData> = Record<string, SessionData>>
extends core.Response<ResBody, Locals> { }
interface Router extends core.Router { }
interface Send extends core.Send { }
}
export = e;
if you notice from the above I added an interface call SessionData if you look almost at the end I set Locals to be equals to it. at the Response
now at yow endPoint you can apply it like this
import type e from "express";
export const endPoint: e.RequestHandler = (req, res, next) => {
//code
};
you can go beong that you can also add them params if there are any, the res body, the req body and stuff
const endPoint: e.RequestHandler<YowParamsObj,YowResBodyObj,YowReqBodyObj,ThemQueryParamsObj> = (req, res, next) => {
//code
};

TypeError [ERR_UNKNOWN_FILE_EXTENSION]:

This is the command i'm trying to run ./bitgo-express --port 3080 --env test --bind localhost and I'm receiving following error:
(node:367854) ExperimentalWarning: The ESM module loader is experimental.
internal/process/esm_loader.js:90
internalBinding('errors').triggerUncaughtException(
^
TypeError [ERR_UNKNOWN_FILE_EXTENSION]: Unknown file extension "" for /home/root/example.com/BitGoJS/modules/express/bin/bitgo-express
at Loader.defaultGetFormat [as _getFormat] (internal/modules/esm/get_format.js:65:15)
at Loader.getFormat (internal/modules/esm/loader.js:116:42)
at Loader.getModuleJob (internal/modules/esm/loader.js:247:31)
at processTicksAndRejections (internal/process/task_queues.js:97:5)
at async Loader.import (internal/modules/esm/loader.js:181:17)
at async Object.loadESM (internal/process/esm_loader.js:84:5) {
code: 'ERR_UNKNOWN_FILE_EXTENSION'
}
bitgo-express:
#!/usr/bin/env node
const { init } = require('../src/expressApp.ts');
if (require.main === module) {
init()
.catch(err => {
console.log(`Fatal error: ${err.message}`);
console.log(err.stack);
});
}
expressApp.ts:
/**
* #prettier
*/
import * as express from 'express';
import * as httpProxy from 'http-proxy';
import * as url from 'url';
import * as Bluebird from 'bluebird';
import * as path from 'path';
import * as _ from 'lodash';
import * as debugLib from 'debug';
import * as https from 'https';
import * as http from 'http';
import * as morgan from 'morgan';
import * as fs from 'fs';
import { Request as StaticRequest } from 'express-serve-static-core';
import { Config, config } from './config';
const debug = debugLib('bitgo:express');
// eslint-disable-next-line #typescript-eslint/camelcase
import { SSL_OP_NO_TLSv1 } from 'constants';
import { IpcError, NodeEnvironmentError, TlsConfigurationError } from './errors';
import { Environments } from 'bitgo';
import { setupRoutes } from './clientRoutes';
const { version } = require('bitgo/package.json');
const pjson = require('../package.json');
const BITGOEXPRESS_USER_AGENT = `BitGoExpress/${pjson.version} BitGoJS/${version}`;
/**
* Set up the logging middleware provided by morgan
*
* #param app
* #param config
*/
function setupLogging(app: express.Application, config: Config): void {
// Set up morgan for logging, with optional logging into a file
let middleware;
if (config.logFile) {
// create a write stream (in append mode)
const accessLogPath = path.resolve(config.logFile);
const accessLogStream = fs.createWriteStream(accessLogPath, { flags: 'a' });
console.log('Log location: ' + accessLogPath);
// setup the logger
middleware = morgan('combined', { stream: accessLogStream });
} else {
middleware = morgan('combined');
}
app.use(middleware);
morgan.token('remote-user', function(req) {
return req.isProxy ? 'proxy' : 'local_express';
});
}
/**
* If we're running in a custom env, set the appropriate environment URI and network properties
*
* #param config
*/
function configureEnvironment(config: Config): void {
const { customRootUri, customBitcoinNetwork } = config;
if (customRootUri) {
Environments['custom'].uri = customRootUri;
}
if (customBitcoinNetwork) {
Environments['custom'].network = customBitcoinNetwork;
}
}
/**
* Create and configure the proxy middleware and add it to the app middleware stack
*
* #param app bitgo-express Express app
* #param config
*/
function configureProxy(app: express.Application, config: Config): void {
const { env, timeout } = config;
// Mount the proxy middleware
const options = {
timeout: timeout,
proxyTimeout: timeout,
secure: true,
};
if (Environments[env].network === 'testnet') {
// Need to do this to make supertest agent pass (set rejectUnauthorized to false)
options.secure = false;
}
const proxy = httpProxy.createProxyServer(options);
const sendError = (res: http.ServerResponse, status: number, json: object) => {
res.writeHead(status, {
'Content-Type': 'application/json',
});
res.end(JSON.stringify(json));
};
proxy.on('proxyReq', function(proxyReq, req) {
// Need to rewrite the host, otherwise cross-site protection kicks in
const parsedUri = url.parse(Environments[env].uri).hostname;
if (parsedUri) {
proxyReq.setHeader('host', parsedUri);
}
const userAgent = req.headers['user-agent']
? BITGOEXPRESS_USER_AGENT + ' ' + req.headers['user-agent']
: BITGOEXPRESS_USER_AGENT;
proxyReq.setHeader('User-Agent', userAgent);
});
proxy.on('error', (err, _, res) => {
debug('Proxy server error: ', err);
sendError(res, 500, {
error: 'BitGo Express encountered an error while attempting to proxy your request to BitGo. Please try again.',
});
});
proxy.on('econnreset', (err, _, res) => {
debug('Proxy server connection reset error: ', err);
sendError(res, 500, {
error:
'BitGo Express encountered a connection reset error while attempting to proxy your request to BitGo. Please try again.',
});
});
app.use(function(req: StaticRequest, res: express.Response) {
if (req.url && (/^\/api\/v[12]\/.*$/.test(req.url) || /^\/oauth\/token.*$/.test(req.url))) {
req.isProxy = true;
proxy.web(req, res, { target: Environments[env].uri, changeOrigin: true });
return;
}
// user tried to access a url which is not an api route, do not proxy
res.status(404).send('bitgo-express can only proxy BitGo API requests');
});
}
/**
* Create an HTTP server configured for accepting HTTPS connections
*
* #param config application configuration
* #param app
* #return {Server}
*/
async function createHttpsServer(
app: express.Application,
config: Config & { keyPath: string; crtPath: string }
): Promise<https.Server> {
const { keyPath, crtPath } = config;
const privateKeyPromise = fs.promises.readFile(keyPath, 'utf8');
const certificatePromise = fs.promises.readFile(crtPath, 'utf8');
const [key, cert] = await Promise.all([privateKeyPromise, certificatePromise]);
// eslint-disable-next-line #typescript-eslint/camelcase
return https.createServer({ secureOptions: SSL_OP_NO_TLSv1, key, cert }, app);
}
/**
* Create an HTTP server configured for accepting plain old HTTP connections
*
* #param app
* #return {Server}
*/
function createHttpServer(app: express.Application): http.Server {
return http.createServer(app);
}
/**
* Create a startup function which will be run upon server initialization
*
* #param config
* #param baseUri
* #return {Function}
*/
export function startup(config: Config, baseUri: string): () => void {
return function() {
const { env, ipc, customRootUri, customBitcoinNetwork } = config;
console.log('BitGo-Express running');
console.log(`Environment: ${env}`);
if (ipc) {
console.log(`IPC path: ${ipc}`);
} else {
console.log(`Base URI: ${baseUri}`);
}
if (customRootUri) {
console.log(`Custom root URI: ${customRootUri}`);
}
if (customBitcoinNetwork) {
console.log(`Custom bitcoin network: ${customBitcoinNetwork}`);
}
};
}
/**
* helper function to determine whether we should run the server over TLS or not
*/
function isTLS(config: Config): config is Config & { keyPath: string; crtPath: string } {
const { keyPath, crtPath } = config;
return Boolean(keyPath && crtPath);
}
/**
* Create either a HTTP or HTTPS server
* #param config
* #param app
* #return {Server}
*/
export async function createServer(config: Config, app: express.Application) {
return isTLS(config) ? await createHttpsServer(app, config) : createHttpServer(app);
}
/**
* Create the base URI where the BitGoExpress server will be available once started
* #return {string}
*/
export function createBaseUri(config: Config): string {
const { bind, port } = config;
const tls = isTLS(config);
const isStandardPort = (port === 80 && !tls) || (port === 443 && tls);
return `http${tls ? 's' : ''}://${bind}${!isStandardPort ? ':' + port : ''}`;
}
/**
* Check environment and other preconditions to ensure bitgo-express can start safely
* #param config
*/
function checkPreconditions(config: Config) {
const { env, disableEnvCheck, bind, ipc, disableSSL, keyPath, crtPath, customRootUri, customBitcoinNetwork } = config;
// warn or throw if the NODE_ENV is not production when BITGO_ENV is production - this can leak system info from express
if (env === 'prod' && process.env.NODE_ENV !== 'production') {
if (!disableEnvCheck) {
throw new NodeEnvironmentError(
'NODE_ENV should be set to production when running against prod environment. Use --disableenvcheck if you really want to run in a non-production node configuration.'
);
} else {
console.warn(
`warning: unsafe NODE_ENV '${process.env.NODE_ENV}'. NODE_ENV must be set to 'production' when running against BitGo production environment.`
);
}
}
const needsTLS = !ipc && env === 'prod' && bind !== 'localhost' && !disableSSL;
// make sure keyPath and crtPath are set when running over TLS
if (needsTLS && !(keyPath && crtPath)) {
throw new TlsConfigurationError('Must enable TLS when running against prod and listening on external interfaces!');
}
if (Boolean(keyPath) !== Boolean(crtPath)) {
throw new TlsConfigurationError('Must provide both keypath and crtpath when running in TLS mode!');
}
if ((customRootUri || customBitcoinNetwork) && env !== 'custom') {
console.warn(`customRootUri or customBitcoinNetwork is set, but env is '${env}'. Setting env to 'custom'.`);
config.env = 'custom';
}
}
export function app(cfg: Config): express.Application {
debug('app is initializing');
const app = express();
setupLogging(app, cfg);
const { debugNamespace, disableProxy } = cfg;
// enable specified debug namespaces
if (_.isArray(debugNamespace)) {
_.forEach(debugNamespace, ns => debugLib.enable(ns));
}
checkPreconditions(cfg);
// Be more robust about accepting URLs with double slashes
app.use(function(req, res, next) {
req.url = req.url.replace(/\/\//g, '/');
next();
});
// Decorate the client routes
setupRoutes(app, cfg);
configureEnvironment(cfg);
if (!disableProxy) {
configureProxy(app, cfg);
}
return app;
}
/**
* Prepare to listen on an IPC (unix domain) socket instead of a normal TCP port.
* #param ipcSocketFilePath path to file where IPC socket should be created
*/
export async function prepareIpc(ipcSocketFilePath: string) {
if (process.platform === 'win32') {
throw new IpcError(`IPC option is not supported on platform ${process.platform}`);
}
try {
const stat = fs.statSync(ipcSocketFilePath);
if (!stat.isSocket()) {
throw new IpcError('IPC socket is not actually a socket');
}
// ipc socket does exist and is indeed a socket. However, the socket cannot already exist prior
// to being bound since it will be created by express internally when binding. If there's a stale
// socket from the last run, clean it up before attempting to bind to it again. Arguably, it would
// be better to do this before exiting, but that gets a bit more complicated when all exit paths
// need to clean up the socket file correctly.
fs.unlinkSync(ipcSocketFilePath);
} catch (e) {
if (e.code !== 'ENOENT') {
throw e;
}
}
}
export async function init(): Bluebird<void> {
const cfg = config();
const expressApp = app(cfg);
const server = await createServer(cfg, expressApp);
const { port, bind, ipc } = cfg;
const baseUri = createBaseUri(cfg);
if (ipc) {
await prepareIpc(ipc);
server.listen(ipc, startup(cfg, baseUri));
} else {
server.listen(port, bind, startup(cfg, baseUri));
}
server.timeout = 300 * 1000; // 5 minutes
}
Node.js version: 12.19.1
NPM version: 6.14.8
How can I fix this issue? I saw this SyntaxError: Cannot use import statment without a module, TypeError [ERR_UNKNOWN_FILE_EXTENSION]: Catch 22 which was similar to mine. But that answer didn't help me as then I'll have to edit the whole file and even I did I'm stuck in 38 line. If someone can translate the whole code into node friendly require, Please translate it. Much appreciated!
You can run your script like this:
npx ts-node ./path/to/your-script.ts
You may want to build your scripts with typescript compiler and run the build result as ordinary JS script.
But for that you need to setup some bundler like for example Webpack and you need to configure plugins for handling Typescript compilation for example you can use Bable.

Translate expressApp.ts to a node friendly require

I want to change translate "import" to a node friendly require in following code.
Example: import * as express from 'express'; to const express = require('express');
I'm stucked at import { Request as StaticRequest } from 'express-serve-static-core'; line. Can someone help me, please
I want to translate the whole code to node friendly code
/**
* #prettier
*/
import * as express from 'express';
import * as httpProxy from 'http-proxy';
import * as url from 'url';
import * as Bluebird from 'bluebird';
import * as path from 'path';
import * as _ from 'lodash';
import * as debugLib from 'debug';
import * as https from 'https';
import * as http from 'http';
import * as morgan from 'morgan';
import * as fs from 'fs';
import { Request as StaticRequest } from 'express-serve-static-core';
import { Config, config } from './config';
const debug = debugLib('bitgo:express');
// eslint-disable-next-line #typescript-eslint/camelcase
import { SSL_OP_NO_TLSv1 } from 'constants';
import { IpcError, NodeEnvironmentError, TlsConfigurationError } from './errors';
import { Environments } from 'bitgo';
import { setupRoutes } from './clientRoutes';
const { version } = require('bitgo/package.json');
const pjson = require('../package.json');
const BITGOEXPRESS_USER_AGENT = `BitGoExpress/${pjson.version} BitGoJS/${version}`;
/**
* Set up the logging middleware provided by morgan
*
* #param app
* #param config
*/
function setupLogging(app: express.Application, config: Config): void {
// Set up morgan for logging, with optional logging into a file
let middleware;
if (config.logFile) {
// create a write stream (in append mode)
const accessLogPath = path.resolve(config.logFile);
const accessLogStream = fs.createWriteStream(accessLogPath, { flags: 'a' });
console.log('Log location: ' + accessLogPath);
// setup the logger
middleware = morgan('combined', { stream: accessLogStream });
} else {
middleware = morgan('combined');
}
app.use(middleware);
morgan.token('remote-user', function(req) {
return req.isProxy ? 'proxy' : 'local_express';
});
}
/**
* If we're running in a custom env, set the appropriate environment URI and network properties
*
* #param config
*/
function configureEnvironment(config: Config): void {
const { customRootUri, customBitcoinNetwork } = config;
if (customRootUri) {
Environments['custom'].uri = customRootUri;
}
if (customBitcoinNetwork) {
Environments['custom'].network = customBitcoinNetwork;
}
}
/**
* Create and configure the proxy middleware and add it to the app middleware stack
*
* #param app bitgo-express Express app
* #param config
*/
function configureProxy(app: express.Application, config: Config): void {
const { env, timeout } = config;
// Mount the proxy middleware
const options = {
timeout: timeout,
proxyTimeout: timeout,
secure: true,
};
if (Environments[env].network === 'testnet') {
// Need to do this to make supertest agent pass (set rejectUnauthorized to false)
options.secure = false;
}
const proxy = httpProxy.createProxyServer(options);
const sendError = (res: http.ServerResponse, status: number, json: object) => {
res.writeHead(status, {
'Content-Type': 'application/json',
});
res.end(JSON.stringify(json));
};
proxy.on('proxyReq', function(proxyReq, req) {
// Need to rewrite the host, otherwise cross-site protection kicks in
const parsedUri = url.parse(Environments[env].uri).hostname;
if (parsedUri) {
proxyReq.setHeader('host', parsedUri);
}
const userAgent = req.headers['user-agent']
? BITGOEXPRESS_USER_AGENT + ' ' + req.headers['user-agent']
: BITGOEXPRESS_USER_AGENT;
proxyReq.setHeader('User-Agent', userAgent);
});
proxy.on('error', (err, _, res) => {
debug('Proxy server error: ', err);
sendError(res, 500, {
error: 'BitGo Express encountered an error while attempting to proxy your request to BitGo. Please try again.',
});
});
proxy.on('econnreset', (err, _, res) => {
debug('Proxy server connection reset error: ', err);
sendError(res, 500, {
error:
'BitGo Express encountered a connection reset error while attempting to proxy your request to BitGo. Please try again.',
});
});
app.use(function(req: StaticRequest, res: express.Response) {
if (req.url && (/^\/api\/v[12]\/.*$/.test(req.url) || /^\/oauth\/token.*$/.test(req.url))) {
req.isProxy = true;
proxy.web(req, res, { target: Environments[env].uri, changeOrigin: true });
return;
}
// user tried to access a url which is not an api route, do not proxy
res.status(404).send('bitgo-express can only proxy BitGo API requests');
});
}
/**
* Create an HTTP server configured for accepting HTTPS connections
*
* #param config application configuration
* #param app
* #return {Server}
*/
async function createHttpsServer(
app: express.Application,
config: Config & { keyPath: string; crtPath: string }
): Promise<https.Server> {
const { keyPath, crtPath } = config;
const privateKeyPromise = fs.promises.readFile(keyPath, 'utf8');
const certificatePromise = fs.promises.readFile(crtPath, 'utf8');
const [key, cert] = await Promise.all([privateKeyPromise, certificatePromise]);
// eslint-disable-next-line #typescript-eslint/camelcase
return https.createServer({ secureOptions: SSL_OP_NO_TLSv1, key, cert }, app);
}
/**
* Create an HTTP server configured for accepting plain old HTTP connections
*
* #param app
* #return {Server}
*/
function createHttpServer(app: express.Application): http.Server {
return http.createServer(app);
}
/**
* Create a startup function which will be run upon server initialization
*
* #param config
* #param baseUri
* #return {Function}
*/
export function startup(config: Config, baseUri: string): () => void {
return function() {
const { env, ipc, customRootUri, customBitcoinNetwork } = config;
console.log('BitGo-Express running');
console.log(`Environment: ${env}`);
if (ipc) {
console.log(`IPC path: ${ipc}`);
} else {
console.log(`Base URI: ${baseUri}`);
}
if (customRootUri) {
console.log(`Custom root URI: ${customRootUri}`);
}
if (customBitcoinNetwork) {
console.log(`Custom bitcoin network: ${customBitcoinNetwork}`);
}
};
}
/**
* helper function to determine whether we should run the server over TLS or not
*/
function isTLS(config: Config): config is Config & { keyPath: string; crtPath: string } {
const { keyPath, crtPath } = config;
return Boolean(keyPath && crtPath);
}
/**
* Create either a HTTP or HTTPS server
* #param config
* #param app
* #return {Server}
*/
export async function createServer(config: Config, app: express.Application) {
return isTLS(config) ? await createHttpsServer(app, config) : createHttpServer(app);
}
/**
* Create the base URI where the BitGoExpress server will be available once started
* #return {string}
*/
export function createBaseUri(config: Config): string {
const { bind, port } = config;
const tls = isTLS(config);
const isStandardPort = (port === 80 && !tls) || (port === 443 && tls);
return `http${tls ? 's' : ''}://${bind}${!isStandardPort ? ':' + port : ''}`;
}
/**
* Check environment and other preconditions to ensure bitgo-express can start safely
* #param config
*/
function checkPreconditions(config: Config) {
const { env, disableEnvCheck, bind, ipc, disableSSL, keyPath, crtPath, customRootUri, customBitcoinNetwork } = config;
// warn or throw if the NODE_ENV is not production when BITGO_ENV is production - this can leak system info from express
if (env === 'prod' && process.env.NODE_ENV !== 'production') {
if (!disableEnvCheck) {
throw new NodeEnvironmentError(
'NODE_ENV should be set to production when running against prod environment. Use --disableenvcheck if you really want to run in a non-production node configuration.'
);
} else {
console.warn(
`warning: unsafe NODE_ENV '${process.env.NODE_ENV}'. NODE_ENV must be set to 'production' when running against BitGo production environment.`
);
}
}
const needsTLS = !ipc && env === 'prod' && bind !== 'localhost' && !disableSSL;
// make sure keyPath and crtPath are set when running over TLS
if (needsTLS && !(keyPath && crtPath)) {
throw new TlsConfigurationError('Must enable TLS when running against prod and listening on external interfaces!');
}
if (Boolean(keyPath) !== Boolean(crtPath)) {
throw new TlsConfigurationError('Must provide both keypath and crtpath when running in TLS mode!');
}
if ((customRootUri || customBitcoinNetwork) && env !== 'custom') {
console.warn(`customRootUri or customBitcoinNetwork is set, but env is '${env}'. Setting env to 'custom'.`);
config.env = 'custom';
}
}
export function app(cfg: Config): express.Application {
debug('app is initializing');
const app = express();
setupLogging(app, cfg);
const { debugNamespace, disableProxy } = cfg;
// enable specified debug namespaces
if (_.isArray(debugNamespace)) {
_.forEach(debugNamespace, ns => debugLib.enable(ns));
}
checkPreconditions(cfg);
// Be more robust about accepting URLs with double slashes
app.use(function(req, res, next) {
req.url = req.url.replace(/\/\//g, '/');
next();
});
// Decorate the client routes
setupRoutes(app, cfg);
configureEnvironment(cfg);
if (!disableProxy) {
configureProxy(app, cfg);
}
return app;
}
/**
* Prepare to listen on an IPC (unix domain) socket instead of a normal TCP port.
* #param ipcSocketFilePath path to file where IPC socket should be created
*/
export async function prepareIpc(ipcSocketFilePath: string) {
if (process.platform === 'win32') {
throw new IpcError(`IPC option is not supported on platform ${process.platform}`);
}
try {
const stat = fs.statSync(ipcSocketFilePath);
if (!stat.isSocket()) {
throw new IpcError('IPC socket is not actually a socket');
}
// ipc socket does exist and is indeed a socket. However, the socket cannot already exist prior
// to being bound since it will be created by express internally when binding. If there's a stale
// socket from the last run, clean it up before attempting to bind to it again. Arguably, it would
// be better to do this before exiting, but that gets a bit more complicated when all exit paths
// need to clean up the socket file correctly.
fs.unlinkSync(ipcSocketFilePath);
} catch (e) {
if (e.code !== 'ENOENT') {
throw e;
}
}
}
export async function init(): Bluebird<void> {
const cfg = config();
const expressApp = app(cfg);
const server = await createServer(cfg, expressApp);
const { port, bind, ipc } = cfg;
const baseUri = createBaseUri(cfg);
if (ipc) {
await prepareIpc(ipc);
server.listen(ipc, startup(cfg, baseUri));
} else {
server.listen(port, bind, startup(cfg, baseUri));
}
server.timeout = 300 * 1000; // 5 minutes
}
You can use Destructuring and Rename variable syntax of ES6.
require('express-serve-static-core') returns a object, just do it as a normal object.
const { Request: StaticRequest } = require('express-serve-static-core');

App Engine Node.js: how to link app logs and requests logs

I am using Node.js on App Engine Standard and Flexible.
In the logs viewer, is it possible to display application logs nested inside request logs?
Yes it is possible to correlate application logs and request logs. This is the end result in the Logs Viewer:
To achieve this you can either:
Use both the #google-cloud/trace-agent and #google-cloud/logging-bunyan modules in your application. When you do so, your logs are automatically annotated with the correct Trace ID (see docs for Bunyan).
Extract the trace ID from the request header
If you do not want to use the Trace module, you can extract the trace ID from the request header, use the following code to extract the traceId:
const traceHeader = req && req.headers ? req.headers['x-cloud-trace-context'] || '' : '';
const traceId = traceHeader ? traceHeader.split('/')[0] : '';
Then, you need to populate the trace attribute of your log entries. When using the Bunyan logger, use the following code:
logger.info({
'logging.googleapis.com/trace': `projects/${project}/traces/${traceId}`
}, 'your message');
I also faced the same issue sometime back and did some workaround to make it. But in the above-mentioned solution might not help in some use cases where you have don't req, res object reference.
So here the solution. it will group all the logs under the request log.
Also created -> NPM Module
File Name: correlate-logs.js
import bunyan from 'bunyan';
import { LOGGING_TRACE_KEY } from '#google-cloud/logging-bunyan';
import cls from 'cls-hooked';
import uuid from 'uuid/v1';
/**
* CreateLogger will return loggerContextMiddleware and log.
* Bind the loggerContextMiddleware on top to corelate other middleware logs. `app.use(loggerContextMiddleware);`
* then you can log like this anywhere `log.info('This is helpful to see corelated logs in nodejs.)` and it will show with-in reqeust log.
* #param {*} options
*/
export default function createLogger(projectId, bunyanLoggerOptions) {
if (!projectId || !bunyanLoggerOptions) throw new Error('Please pass the required fields projectId and bunyanLoggerOption');
const ns = cls.createNamespace(`logger/${uuid()}`); // To create unique namespace.
const logger = bunyan.createLogger(bunyanLoggerOptions);
/**
* Express Middleware to add request context to logger for corelating the logs in GCP.
* #param {*} req
* #param {*} res
* #param {*} next
*/
const loggerContextMiddleware = (req, res, next) => {
const traceHeader = (req && req.headers && req.headers['x-cloud-trace-context']) || '';
if (traceHeader) {
ns.bindEmitter(req);
ns.bindEmitter(res);
const traceId = traceHeader ? traceHeader.split('/')[0] : '';
const trace = `projects/${projectId}/traces/${traceId}`;
ns.run(() => {
ns.set('trace', trace);
next();
});
} else {
next();
}
};
/**
* Helper method to get the trace id from CLS hook.
*/
function getTrace() {
if (ns && ns.active) return ns.get('trace');
return '';
}
/**
* Simple wrapper to avoid pushing dev logs to cloud.
* #param {*} level
* #param {*} msg
*/
function printLog(level, ...msg) {
const trace = getTrace();
if (trace) { logger[level]({ [LOGGING_TRACE_KEY]: trace }, ...msg); } else { logger[level](...msg); }
}
/**
* Little wrapper to abstract the log level.
*/
const log = ['trace', 'debug', 'info', 'warn', 'error', 'fatal'].reduce((prev, curr) => ({ [curr]: (...msg) => printLog(curr, ...msg), ...prev }), {});
return { loggerContextMiddleware, log };
}
File Name: logger.js
import { LoggingBunyan } from '#google-cloud/logging-bunyan';
import createLogger from '../lib/corelate-logs';
import { getProjectId, ifDev } from './config';
// Creates a Bunyan Stackdriver Logging client
const loggingBunyan = new LoggingBunyan();
let loggerOption;
if (ifDev()) {
const bunyanDebugStream = require('bunyan-debug-stream'); // eslint-disable-line
loggerOption = {
name: 'my-service',
streams: [{
level: 'info',
type: 'raw',
stream: bunyanDebugStream({
forceColor: true,
}),
}],
serializers: bunyanDebugStream.serializers,
};
} else {
loggerOption = {
name: 'my-service',
level: 'info',
streams: [loggingBunyan.stream('info')],
};
}
const { loggerContextMiddleware, log } = createLogger(getProjectId() || 'dev', loggerOption);
export { loggerContextMiddleware, log };
Hope this helps somebody.

Injecting App as a parameter vs Import App from server.js in loopBack framework

I'm using loopback3 and I'm thinking about optimising the codebase now, what is the point of injecting the application object as a parameter to getUser method from afterRemote hook https://loopback.io/doc/en/lb2/Remote-hooks.html, if I can access loopback instance by requiring it directly inside userService.ts.
I feel that I'm missing something important here, I tried to summarise below
A. For me importing is better since I will have less code and, there
will be no need to inject the app each time.
B. Injected and Imported
objects are equal, _.isEqual(app, App)
C. I've checked the
performance with process.hrtime() and got same results.
app/models/activity.ts
import {UserService} from 'app/service/userService';
import {Attach} from 'app/models/remote/activityRemote';
export = function (Activity) {
Activity.afterRemote('find', function (ctx, result, next) {
UserService.getUser(Activity.app, Activity.username)
.then(() => next())
.catch(next);
});
/**
* attach remote
*/
Attach(Activity);
};
userService.ts
import {Server} from 'app/server/interface/server';
import * as App from 'app/server/server';
import * as _ from 'lodash';
/**
* #class UserService
*/
export class UserService {
/**
* get user's public profile
* #param {Server} app loopback app
* #param {string} username
* #returns {Promise<User>}
*/
public static getUser(app: Server, username: string): Promise<User> {
return App.models.user.findOne(filter) // Equal and
return app.models.user.findOne(filter) // does the same
.then((user: User) => {
if (!user) {
return Promise.reject(ServerError.get('User not found', 404));
}
return Promise.resolve(user);
});
}
}
server.ts
import {Server} from 'app/server/interface/server';
import * as loopback from 'loopback';
import * as boot from 'loopback-boot';
let App: Server = loopback();
module.exports = App;
export = App;
App.start = () => {
return App.listen(() => {
const baseUrl = App.get('url').replace(/\/$/, '');
App.emit('started');
console.log('Web server listening at: %s', baseUrl);
if (App.get('loopback-component-explorer')) {
console.log(
'Browse your REST API at %s%s',
baseUrl,
App.get('loopback-component-explorer').mountPath
);
}
});
};
boot(App, __dirname, (err: Error) => {
if (err) {
throw err;
}
if (require.main === module) {
App.start();
}
});
Unless you have multiple loopback applications in the same process going around, then you have no reason to pass the application as a parameter. Just import the app as it's more readable and cleaner.
Also you don't need to use Promise.resolve when you already have a promise:
return app.models.user.findOne(filter) // does the same
.then((user: User) => {
if (!user) {
throw ServerError.get('User not found', 404);
}
return user;
});
This will have the same effect.

Resources