How to log GRPC request and response in Nestjs? - node.js

I am trying to log the grpc request and response in my nestjs project. It seems that the middleware would help me do that. However, the middleware in nesjs is for HTTP only. So, is there any other way that can be used to solve my problem? Maybe interceptors?

Based on the examples on this github. I used the following code to log the request/response and redact the logs for a gRPC client:
// users.module.ts
#Module({
imports: [
ConfigModule.forRoot(),
ClientsModule.registerAsync([
{
name: USERS_PACKAGE,
imports: [ConfigModule],
useFactory: async (configService: ConfigService) => ({
transport: Transport.GRPC,
options: {
package: 'users',
credentials: credentials.createInsecure(),
protoPath: join(__dirname, './proto/users.proto'),
url: configService.get<string>('services.users.url'),
loader: { keepCase: true },
channelOptions: {
interceptors: [
(options, nextCall) => {
const logger: Logger = new Logger('UsersModule');
const redactedFields: string[] = configService.get<string[]>('logger.redact.fields');
const path: string = options.method_definition.path;
logger.verbose('-------------------------------------- GRPC CALL ----------------------------------');
const interceptingCall = new InterceptingCall(
nextCall(options),
GrpcLoggingInterceptor(path, redactedFields),
);
return interceptingCall;
},
],
},
},
}),
inject: [ConfigService],
},
]),
],
providers: [GetUsersClient],
exports: [GetUsersClient],
})
export class UsersModule {}
The grpc logging interceptor to intercept the grpc requests
// grpc-logging.interceptor.ts
import { Logger } from '#nestjs/common';
export const GrpcLoggingInterceptor = (path: string, redactedFields: string[]) => {
const logger: Logger = new Logger('GrpcLoggingInterceptor');
function redact(data: string): string {
return JSON.stringify(data, (k, v) => (redactedFields.includes(k) ? `[Redacted]` : v));
}
return {
start: function (metadata, listener, next) {
const newListener = {
onReceiveMetadata: function (metadata, next) {
logger.verbose(`response metadata : ${redact(metadata)}`);
next(metadata);
},
onReceiveMessage: function (message, next) {
logger.verbose(`response body : ${redact(message)}`);
next(message);
},
onReceiveStatus: function (status, next) {
logger.verbose(`response status : ${redact(status)}`);
next(status);
},
};
next(metadata, newListener);
},
sendMessage: function (message, next) {
logger.verbose(`path: ${JSON.stringify(path)}`);
logger.verbose(`request body : ${redact(message)}`);
next(message);
},
};
};
The fields to redact:
// config.yaml
logger:
redact:
fields:
- token
- password
- address
- telephone

You can use an Interceptor. In fact, that's exactly what my OgmaInterceptor does, and there's even a plugin for gRPC. In general, just something simple like
#Injectable()
export class LoggingInterceptor {
intercept(context: ExecutionContext, next: CallHandler) {
const start = Date.now()
console.log('Request start');
return next.handle().pipe(
tap((data) => {
console.log(`Request took ${Date.now() - start}ms. Response length: ${Buffer.from(JSON.stringify(data) ?? '').length} bytes`);
})
);
}
}
would be a simple first pass at writing something for this

Related

How to mock Jest Redis in Nestjs

I'm working on NestJs application and wrote unit test for my authenticateUser function in user.service.ts.It's has pass in my local machine.but when I deployed it in to server and run unit test, i got an error Redis connection to 127.0.0.1:6379 failed - connect ECONNREFUSED.Seems like redis mock is not working.How should I mock redis and resolve this issue for working?
user.service.ts
async authenticateUser(authDto: AuthDTO): Promise<AuthResponse> {
try {
const userData = await this.userRepository.findOne({msisdn});
if(userData){
await this.redisCacheService.setCache(msisdn, userData);
}
} catch (error) {
console.log(error)
}
}
redisCache.service.ts
export class RedisCacheService {
constructor(
#Inject(CACHE_MANAGER) private readonly cache: Cache,
) {}
async setCache(key, value) {
await this.cache.set(key, value);
}
}
user.service.spec.ts
describe('Test User Service', () => {
let userRepository: Repository<UserEntity>;
let userService: UserService;
let redisCacheService: RedisCacheService;
let cacheManager: any;
beforeEach(async () => {
const module: TestingModule = await Test.createTestingModule({
providers: [
UserService,
UserEntity,
RedisCacheService,
{
provide: getRepositoryToken(UserEntity),
useClass: registeredApplicationRepositoryMockFactory,
},
],
imports: [CacheModule.register({})],
}).compile();
userService = module.get<UserService>(UserService);
userRepository = module.get<Repository<UserEntity>>(
getRepositoryToken(UserEntity),
);
redisCacheService = module.get<RedisCacheService>(RedisCacheService);
cacheManager = module.get<any>(CACHE_MANAGER);
});
it('authenticateUser should return success response', async () => {
const userEntity = { id: 1, name: 'abc', age: 25 };
const mockSuccessResponse = new AuthResponse(
HttpStatus.OK,
STRING.SUCCESS,
`${STRING.USER} ${STRING.AUTHENTICATE} ${STRING.SUCCESS}`,
{},
);
jest.spyOn(userRepository, 'findOne').mockResolvedValueOnce(userEntity);
jest.spyOn(redisCacheService, 'setCache').mockResolvedValueOnce(null);
expect(await userService.authenticateUser(mockAuthBody)).toEqual(mockSuccessResponse);
});
});
You can mock CACHE_MANAGER using a custom provider:
import { CACHE_MANAGER } from '#nestjs/common';
import { Cache } from 'cache-manager';
describe('AppService', () => {
let service: AppService;
let cache: Cache;
beforeEach(async () => {
const app = await Test.createTestingModule({
providers: [
AppService,
{
provide: CACHE_MANAGER,
useValue: {
get: () => 'any value',
set: () => jest.fn(),
},
},
],
})
.compile();
service = app.get<AppService>(AppService);
cache = app.get(CACHE_MANAGER);
});
// Then you can use jest.spyOn() to spy and mock
it(`should cache the value`, async () => {
const spy = jest.spyOn(cache, 'set');
await service.cacheSomething();
expect(spy).toHaveBeenCalledTimes(1);
expect(spy.mock.calls[0][0]).toEqual('key');
expect(spy.mock.calls[0][1]).toEqual('value');
});
it(`should get the value from cache`, async () => {
const spy = jest.spyOn(cache, 'get');
await service.getSomething();
expect(spy).toHaveBeenCalledTimes(1);
});
it(`should return the value from the cache`, async () => {
jest.spyOn(cache, 'get').mockResolvedValueOnce('value');
const res = await service.getSomething();
expect(res).toEqual('value');
}),
});
More details on Custom Providers: https://docs.nestjs.com/fundamentals/custom-providers
Two more things, for unit testing you shouldn't import modules but mock the dependencies instead. And as Daniel said, UserService is not using CACHE_MANAGER but RedisCacheService, so you should mock RedisCacheService.
Usually the best thing to do is to only provide the service you're testing and mock the dependencies.
in order to use the jest spy functions you need to return the jest function directly.
providers: [
AppService,
{
provide: CACHE_MANAGER,
useValue: {
get: () => 'any value',
set: jest.fn(),
},
},
],

What is the best way to handle the react GQL configurations with all query and mutation policies and for file upload?

Need to know the best practice to maintain the react GQL configurations settings where we set all query, mutation, or other policies, and the better way to handle file upload setting?
There are many approaches with different use cases. The following code might be helpful in your case.
import {
ApolloClient,
ApolloLink,
DefaultOptions,
InMemoryCache,
} from "#apollo/client";
import { onError } from "#apollo/client/link/error";
import { createUploadLink } from "apollo-upload-client";
const authMiddleware = new ApolloLink((operation: any, forward: any) => {
const token = localStorage.getItem("token") || null;
operation.setContext({
headers: {
authorization: `Bearer ${token}`,
},
});
return forward(operation);
});
const defaultOptions: DefaultOptions = {
watchQuery: {
fetchPolicy: "cache-and-network",
errorPolicy: "ignore",
},
query: {
fetchPolicy: "network-only",
errorPolicy: "all",
},
mutate: {
errorPolicy: "all",
},
};
const errorLink = onError(
({ graphQLErrors, networkError, operation, forward }: any) => {
// You can modify do something with the errors
}
);
const client = new ApolloClient({
cache: new InMemoryCache(),
connectToDevTools: true,
link: ApolloLink.from([
errorLink,
authMiddleware,
createUploadLink({
uri: process.env.REACT_APP_GRAPHQL_URI || "http://localhost:3001/graphql",
}),
]),
defaultOptions: defaultOptions,
});
export default client;

Looking to use delegateToSchema in nestjs resolver

I have a local schema and a remote schema.
The local schema contains a subset of the remote schema and should proxy these to the remote graphql server.
Normally i would use something like:
public async getCar(
obj: any,
args: any,
context: any,
info: any
): Promise<Car> {
return info.mergeInfo.delegateToSchema({
args: {
id: context.userId,
},
context: {},
fieldName: 'car',
info,
operation: 'query',
schema: carExecutableSchema,
transforms: [],
});
}
but the mergeInfo is not available on the info field.
To get the remote schema I'm using:
const fetchRemoteSchema = async (): Promise<GraphQLSchema> => {
const http = new HttpLink({
fetch: nodeFetch,
uri: process.env.CAR_GRAPHQL_URL,
});
const link = setContext((request, previousContext) => ({
headers: {
Authorization: `Basic ${process.env.CAR_BASIC_AUTH_TOKEN}`,
},
})).concat(http);
const carSchema = await introspectSchema(link);
const executableSchema = makeRemoteExecutableSchema({
link,
schema: carSchema,
});
return executableSchema;
};
and i use this in the module to initialize graphql
GraphQLModule.forRootAsync({
imports: [CarModule],
inject: [CarService],
useFactory: (mainAppService: MainAppService) => ({
context: async ({ req }) => {
return {
userId: 1
};
},
definitions: {
outputAs: 'interface',
path: '../../myschema.gql',
},
typePaths: ['./**/*.graphql'],
}),
}),
How could i make mergeInfo available? Or is there an easier way to delegate a graphql request?
info.mergeInfo is only available when your schema is created with mergeSchemas([]). You can import delegateToSchema as function though: import { delegateToSchema } from "apollo-server-express";

How to change a Database connection dynamically with Request Scope Providers in Nestjs?

Working on a project with Nestjs 6.x, Mongoose, Mongo, etc...
Regarding to the Back End, in my use case, I must change the connection of one of my databases depending of some conditions/parameters coming from some requests.
Basically, I have this
mongoose.createConnection('mongodb://127.0.0.1/whatever-a', { useNewUrlParser: true })
and I want to change to, for example
mongoose.createConnection('mongodb://127.0.0.1/whatever-b', { useNewUrlParser: true })
Therefore, I have in Nestjs the first provider
export const databaseProviders = [
{
provide: 'DbConnectionToken',
useFactory: async (): Promise<typeof mongoose> =>
await mongoose.createConnection('mongodb://127.0.0.1/whatever', { useNewUrlParser: true })
}
I was researching for a while and I found out that in release Nestjs 6.x there are provider requests allowing me to modify dynamically Per-request the injection of some providers.
Anyway, I don't know how to achieve my change neither if it is going to be working in case I'd achieve that
Can anyone help or guide me?
Many thanks in advance.
You can do the following using Nest's built-in Mongoose package:
/*************************
* mognoose.service.ts
*************************/
import { Inject, Injectable, Scope } from '#nestjs/common';
import { MongooseOptionsFactory, MongooseModuleOptions } from '#nestjs/mongoose';
import { REQUEST } from '#nestjs/core';
import { Request } from '#nestjs/common';
#Injectable({ scope: Scope.REQUEST })
export class MongooseConfigService implements MongooseOptionsFactory {
constructor(
#Inject(REQUEST) private readonly request: Request,) {
}
createMongooseOptions(): MongooseModuleOptions {
return {
uri: request.params.uri, // Change this to whatever you want; you have full access to the request object.
};
}
}
/*************************
* mongoose.module.ts
*************************/
import { Module } from '#nestjs/common';
import { MongooseModule } from '#nestjs/mongoose';
import { MongooseConfigService } from 'mognoose.service';
#Module({
imports: [
MongooseModule.forRootAsync({
useClass: MongooseConfigService,
}),
]
})
export class DbModule {}
Then, you can attach whatever you want to the request and change the database per request; hence the use of the Scope.REQUEST. You can read more about Injection Scopes on their docs.
Edit: If you run into issues with PassportJS (or any other package) or the request is empty, it seems to be an error that relates to PassportJS (or the other package) not supporting request scopes; you may read more about the issue on GitHub regarding PassportJS.
I did a simple implementation for nest-mongodb,
The main changes are in mongo-core.module.ts where I store the connections in a map and used them if available instead of creating a new connection every time.
import {
Module,
Inject,
Global,
DynamicModule,
Provider,
OnModuleDestroy,
} from '#nestjs/common';
import { ModuleRef } from '#nestjs/core';
import { MongoClient, MongoClientOptions } from 'mongodb';
import {
DEFAULT_MONGO_CLIENT_OPTIONS,
MONGO_MODULE_OPTIONS,
DEFAULT_MONGO_CONTAINER_NAME,
MONGO_CONTAINER_NAME,
} from './mongo.constants';
import {
MongoModuleAsyncOptions,
MongoOptionsFactory,
MongoModuleOptions,
} from './interfaces';
import { getClientToken, getContainerToken, getDbToken } from './mongo.util';
import * as hash from 'object-hash';
#Global()
#Module({})
export class MongoCoreModule implements OnModuleDestroy {
constructor(
#Inject(MONGO_CONTAINER_NAME) private readonly containerName: string,
private readonly moduleRef: ModuleRef,
) {}
static forRoot(
uri: string,
dbName: string,
clientOptions: MongoClientOptions = DEFAULT_MONGO_CLIENT_OPTIONS,
containerName: string = DEFAULT_MONGO_CONTAINER_NAME,
): DynamicModule {
const containerNameProvider = {
provide: MONGO_CONTAINER_NAME,
useValue: containerName,
};
const connectionContainerProvider = {
provide: getContainerToken(containerName),
useFactory: () => new Map<any, MongoClient>(),
};
const clientProvider = {
provide: getClientToken(containerName),
useFactory: async (connections: Map<any, MongoClient>) => {
const key = hash.sha1({
uri: uri,
clientOptions: clientOptions,
});
if (connections.has(key)) {
return connections.get(key);
}
const client = new MongoClient(uri, clientOptions);
connections.set(key, client);
return await client.connect();
},
inject: [getContainerToken(containerName)],
};
const dbProvider = {
provide: getDbToken(containerName),
useFactory: (client: MongoClient) => client.db(dbName),
inject: [getClientToken(containerName)],
};
return {
module: MongoCoreModule,
providers: [
containerNameProvider,
connectionContainerProvider,
clientProvider,
dbProvider,
],
exports: [clientProvider, dbProvider],
};
}
static forRootAsync(options: MongoModuleAsyncOptions): DynamicModule {
const mongoContainerName =
options.containerName || DEFAULT_MONGO_CONTAINER_NAME;
const containerNameProvider = {
provide: MONGO_CONTAINER_NAME,
useValue: mongoContainerName,
};
const connectionContainerProvider = {
provide: getContainerToken(mongoContainerName),
useFactory: () => new Map<any, MongoClient>(),
};
const clientProvider = {
provide: getClientToken(mongoContainerName),
useFactory: async (
connections: Map<any, MongoClient>,
mongoModuleOptions: MongoModuleOptions,
) => {
const { uri, clientOptions } = mongoModuleOptions;
const key = hash.sha1({
uri: uri,
clientOptions: clientOptions,
});
if (connections.has(key)) {
return connections.get(key);
}
const client = new MongoClient(
uri,
clientOptions || DEFAULT_MONGO_CLIENT_OPTIONS,
);
connections.set(key, client);
return await client.connect();
},
inject: [getContainerToken(mongoContainerName), MONGO_MODULE_OPTIONS],
};
const dbProvider = {
provide: getDbToken(mongoContainerName),
useFactory: (
mongoModuleOptions: MongoModuleOptions,
client: MongoClient,
) => client.db(mongoModuleOptions.dbName),
inject: [MONGO_MODULE_OPTIONS, getClientToken(mongoContainerName)],
};
const asyncProviders = this.createAsyncProviders(options);
return {
module: MongoCoreModule,
imports: options.imports,
providers: [
...asyncProviders,
clientProvider,
dbProvider,
containerNameProvider,
connectionContainerProvider,
],
exports: [clientProvider, dbProvider],
};
}
async onModuleDestroy() {
const clientsMap: Map<any, MongoClient> = this.moduleRef.get<
Map<any, MongoClient>
>(getContainerToken(this.containerName));
if (clientsMap) {
await Promise.all(
[...clientsMap.values()].map(connection => connection.close()),
);
}
}
private static createAsyncProviders(
options: MongoModuleAsyncOptions,
): Provider[] {
if (options.useExisting || options.useFactory) {
return [this.createAsyncOptionsProvider(options)];
} else if (options.useClass) {
return [
this.createAsyncOptionsProvider(options),
{
provide: options.useClass,
useClass: options.useClass,
},
];
} else {
return [];
}
}
private static createAsyncOptionsProvider(
options: MongoModuleAsyncOptions,
): Provider {
if (options.useFactory) {
return {
provide: MONGO_MODULE_OPTIONS,
useFactory: options.useFactory,
inject: options.inject || [],
};
} else if (options.useExisting) {
return {
provide: MONGO_MODULE_OPTIONS,
useFactory: async (optionsFactory: MongoOptionsFactory) =>
await optionsFactory.createMongoOptions(),
inject: [options.useExisting],
};
} else if (options.useClass) {
return {
provide: MONGO_MODULE_OPTIONS,
useFactory: async (optionsFactory: MongoOptionsFactory) =>
await optionsFactory.createMongoOptions(),
inject: [options.useClass],
};
} else {
throw new Error('Invalid MongoModule options');
}
}
}
Check out the full implementation

How to unit test Controller and mock #InjectModel in the Service constructor

I am getting issues while unit testing my controller and getting an error "Nest can't resolve dependencies of my service".
For maximum coverage I wanted to unit test controller and respective services and would like to mock external dependencies like mongoose connection. For the same I already tried suggestions mentioned in the below link but didn't find any luck with that:
https://github.com/nestjs/nest/issues/194#issuecomment-342219043
Please find my code below:
export const deviceProviders = [
{
provide: 'devices',
useFactory: (connection: Connection) => connection.model('devices', DeviceSchema),
inject: ['DbConnectionToken'],
},
];
export class DeviceService extends BaseService {
constructor(#InjectModel('devices') private readonly _deviceModel: Model<Device>) {
super();
}
async getDevices(group): Promise<any> {
try {
return await this._deviceModel.find({ Group: group }).exec();
} catch (error) {
return Promise.reject(error);
}
}
}
#Controller()
export class DeviceController {
constructor(private readonly deviceService: DeviceService) {
}
#Get(':group')
async getDevices(#Res() response, #Param('group') group): Promise<any> {
try {
const result = await this.deviceService.getDevices(group);
return response.send(result);
}
catch (err) {
return response.status(422).send(err);
}
}
}
#Module({
imports: [MongooseModule.forFeature([{ name: 'devices', schema: DeviceSchema }])],
controllers: [DeviceController],
components: [DeviceService, ...deviceProviders],
})
export class DeviceModule { }
Unit test:
describe('DeviceController', () => {
let deviceController: DeviceController;
let deviceService: DeviceService;
const response = {
send: (body?: any) => { },
status: (code: number) => response,
};
beforeEach(async () => {
const module = await Test.createTestingModule({
controllers: [DeviceController],
components: [DeviceService, ...deviceProviders],
}).compile();
deviceService = module.get<DeviceService>(DeviceService);
deviceController = module.get<DeviceController>(DeviceController);
});
describe('getDevices()', () => {
it('should return an array of devices', async () => {
const result = [{
Group: 'group_abc',
DeviceId: 'device_abc',
},
{
Group: 'group_xyz',
DeviceId: 'device_xyz',
}];
jest.spyOn(deviceService, 'getDevices').mockImplementation(() => result);
expect(await deviceController.getDevices(response, null)).toBe(result);
});
});
});
When I am running my test case above, I am getting two errors:
Nest can't resolve dependencies of the DeviceService (?). Please make sure that the argument at index [0] is available in the current context.
Cannot spyOn on a primitive value; undefined given
Example code:
import { Test } from '#nestjs/testing';
import { getModelToken } from '#nestjs/mongoose';
describe('auth', () => {
let deviceController: DeviceController;
let deviceService: DeviceService;
const mockRepository = {
find() {
return {};
}
};
beforeAll(async () => {
const module = await Test.createTestingModule({
imports: [DeviceModule]
})
.overrideProvider(getModelToken('Auth'))
.useValue(mockRepository)
.compile();
deviceService = module.get<DeviceService>(DeviceService);
});
// ...
});
You are not injecting the correct token here. Instead of a plain string you have to use the function getModelToken.
import { getModelToken } from '#nestjs/mongoose';
// ...
{ provide: getModelToken('devices'), useFactory: myFactory },
Here is the solution provided by this repo. See the mongo-sample. I am testing my API using the #injectModel and another service. Here's the snippet:
import { CategoriesService } from './../categories/categories.service';
import { getModelToken } from '#nestjs/mongoose';
import { Test, TestingModule } from '#nestjs/testing';
import { ProductsService } from './products.service';
describe('ProductsService', () => {
let service: ProductsService;
beforeAll(async () => {
const module: TestingModule = await Test.createTestingModule({
// getModelToken to mock the MongoDB connection
providers: [
ProductsService,
CategoriesService,
{
provide: getModelToken('Product'),
useValue: {
find: jest.fn(),
findOne: jest.fn(),
findByIdAndUpdate: jest.fn(),
findByIdAndRemove: jest.fn(),
save: jest.fn(),
},
},
{
provide: getModelToken('Category'),
useValue: {
find: jest.fn(),
findOne: jest.fn(),
findByIdAndUpdate: jest.fn(),
findByIdAndRemove: jest.fn(),
save: jest.fn(),
},
},
],
}).compile();
service = module.get<ProductsService>(ProductsService);
});
// your test case
});

Resources