I'm adding a socket.io "chat" implementation to our NestJS app, currently serving a range of HTTP REST APIs. We have fairly complex tenant-based auth using guards for our REST APIs. Users can belong to one or many tenants, and they target a given tenats via the API URL, which can be either subdomain or path-based, depending on the deployment environment, for example:
//Subdomain based
https://tenant1.api.server.com/endpoint
https://tenant2.api.server.com/endpoint
//Path based
https://api.server.com/tenant1/endpoint
https://api.server.com/tenant2/endpoint
This all works fine for REST APIs, allowing us to determine the intended tenant (and validate the user access to that tenant) within guards.
The new socket.io implementation is being exposed on the same port at endpoint "/socket", meaning that possible full paths for connection could be:
https://tenant1.api.server.com/socket
https://api.server.com/tenant1/socket
Ideally I want to validate the user (via JWT) and the access to the group during the connection of the websocket (and if they are not validated they get immediately disconnected). I have been struggling to implement with guards, so I have done JWT/user validate in the socket gateway, which works ok. For the tenant validation, as per the above, I need the FULL URL that was used for the connection, because I will either be looking at the subdomain OR the path, depending on the deployment. I can get the host from the client handshake headers, but cannot find any way to get at the path. Is there a way to get the full path either from the handshake, or perhaps from Nest? I think perhaps I am limited in what I have access to in the handleConnection method when implementing OnGatewayConnection.
Code so far:
#WebSocketGateway({
namespace: 'socket',
cors: {
origin: '*',
},
})
export class ChannelsGateway
implements OnGatewayInit, OnGatewayConnection, OnGatewayDisconnect {
#WebSocketServer() public server: Server
//Init using separate socket service (allowing for other components to push messages)
afterInit(server: Server) {
this.socketService.socket = server
Logger.debug('Socket.io initialized')
}
//Method for handling the client initial connection
async handleConnection(client: Socket, ...args: any[]) {
//This line gets me the host, such as tenant1.api.server.com but without the path
const host = client.handshake.headers.host
//Get bearer token from authorizaton header and validate
//Disconnect and return if not validated
const bearerJwt = client.handshake.headers.authorization
const decodedToken = await validateJwt(bearerJwt).catch(error => {
client.disconnect()
})
if (!decodedToken) {
return
}
//What can I use to get at the path, such as:
//api.server.com/tenant1/socket
//tenant1.api.server.com/socket
//Then I can extract the "tenant1" with existing code and validate access
//Else disconnect the client
//Further code to establish a private room for the user, etc...
}
//other methods for receiving messages, etc...
}
Related
We are using nestjs for lambda which connects with mongodb for data. We are using nestjs mongoose module. However on deployment for each invocation a new set of connection are made and the previous ones are not released.
We are using forRootAsync
MongooseModule.forRootAsync({
useClass: MongooseConfigService,
})
service looks like this:
#Injectable({ scope: Scope.REQUEST })
export class MongooseConfigService implements MongooseOptionsFactory {
constructor(#Inject(REQUEST) private readonly request: Request) {}
async createMongooseOptions(): Promise<MongooseModuleOptions> {
if (Buffer.isBuffer(this.request.body)) {
this.request.body = JSON.parse(this.request.body.toString());
}
const { db } = this.request.body;
console.log('connecting database', db);
return {
uri: process.env.MONGO_URL,
dbName: db || '',
};
}
}
I understand we need to reuse the same connection. We have achieved it in nodejs by simply checking if the connection already exists and if it does not connect again. Not sure how to achieve the same in nest.js
tried changing the scope of service to Scope.DEFAULT but that didn't help.
I would suggest that you make a connection proxy for MongoDB. Ever time a lambda gets invoked, it will open up a new connection to MongoDB. AWS generally provides a service that allows you to proxy requests through one connect, this is a common issue with RDS etc.
This may help you though: https://www.mongodb.com/docs/atlas/manage-connections-aws-lambda/
We were able to resolve the issue by using a durable provider. NestJs documentation we created a strategy at the root that would use a parameter coming in each request. NestJs would than call the connection module only when a new connection was required.
Note: When you use durable providers, Requests doesn't have all the parameters anymore and now only has the tenantId as per the example.
I have a nestjs application that consumes third party API for data. In order to use that third party API, I need to pass along an access token. This access token is application-wide and not attached to any one user.
What would be the best place to store such a token in Nestjs, meeting the following requirements:
It must be available in the application and not per given user
It must not be exposed to the frontend application
It must work in a load balancer setup
I am looking at Nestjs caching https://docs.nestjs.com/techniques/caching, but I am not sure whether that's the best practice and if it is - should I use it with in-memory storage or something like redis.
Thank you.
If you're working with Load Balancing, then in-memory solutions are dead on arrival, as they will only affect one instance of your server, not all of them. Your best bet for speed purposes and accessibility would be Redis, saving the token under a simple key and keeping it alive from there (and updating it as necessary). Just make sure all your instances connect to the same Redis instance, and that your instance can handle it, shouldn't be a problem, more of a callout
I used a custom provider. Nest allows you to load async custom providers.
export const apiAuth = {
provide: 'API_AUTH',
useFactory: async (authService: AuthService) => {
return await authService.createOrUpdateAccessToken()
},
inject: [AuthService]
}
and below is my api client.
#Injectable()
export class ApiClient {
constructor(#Inject('API_AUTH') private auth: IAuth, private authService: AuthService) { }
public async getApiClient(storeId: string): Promise<ApiClient> {
if (((Date.now() - this.auth.createdAt.getTime()) > ((this.auth.expiresIn - 14400) * 1000))) {
this.auth = await this.authService.createOrUpdateAccessToken()
}
return new ApiClient(storeId, this.auth.accessToken);
}
}
This way token is requested from storage once and lives with the application, when expired token is re-generated and updated.
I have a few Zeit micro services. This setup is a RESTful API for multiple frontends/domains/clients
I need to, in my configs that are spread throughout the apps, differentiate between these clients. I can, in my handlers, setup a process.env.CLIENT_ID for example that I can use in my config handler to know which config to load. However this would mean launching a new http/micro process for each requesting domain (or whatever method I use - info such as client id will prob come in a header) in order to maintain the process.env.CLIENT_ID throughout the request and not have it overwritten by another simultaneous request from another client.
So I have to have each microservice check the client ID, determine if it has already launched a process for that client and use that else launch a new one.
This seems messy but not sure how else to handle things. Passing the client id around with code calls (i.e. getConfg(client, key) is not practical in my situation and I would like to avoid that.
Options:
Pass client id around everywhere
Launch new process per host
?
Is there a better way or have I made a mistake in my assumptions?
If the process per client approach is the better way I am wondering if there is an existing solution to manage this? Ive looked at http proxy, micro cluster etc but none seem to provide a solution to this issue.
Well I found this nice tool https://github.com/othiym23/node-continuation-local-storage
// Micro handler
const { createNamespace } = require('continuation-local-storage')
let namespace = createNamespace('foo')
const handler = async (req, res) => {
const clientId = // some header thing or host
namespace.run(function() {
namespace.set('clientId', clientId)
someCode()
})
})
// Some other file
const { getNamespace } = require('continuation-local-storage')
const someCode = () => {
const namespace = getNamespace('foo')
console.log(namespace.get('clientId'))
}
Ok, this is not what you think it is, I am not asking for help with the async/wait pattern or asynchronous programming I am well versed with those. I am however querying whether something is possible within a Node.JS Express service.
The Scenario
I have a web service which is developed in Node.JS and uses Express.JS to expose some REST endpoints that a client can connect to and send a POST request. For the most part these are Synchronous and will create a SOAP message and send that on to an external service and receive an immediate response which can then be returned to the client, all really simple stuff which is already implemented. So what's your point I hear you say, I am coming to that.
I have a couple of POST interactions that will build a SOAP message to send to an Asynchronous external endpoint where the response will be received asynchronously through an inbound endpoint.
Option 1: What I am looking for in these cases is to be able to build the SOAP message, create a listener (so I can listen for the response to my request), and then send the request to the external service which immediately returns a 200.
Option 2: When I setup the service I want to also setup and listen for incoming requests from the external service whilst also listening for REST requests from the internal service.
The Question
Is either option possible in Node and Express? and, if so how would one achieve this?
NOTE: I know its possible in C# using WCF or a Listener but I would like to avoid this and use Node.JS so any help would be greatly appreciated.
First of all check node-soap if it fits your needs.
Option 1: What I am looking for in these cases is to be able to build the SOAP message, create a listener (so I can listen for the response to my request), and then send the request to the external service which immediately returns a 200.
Here's a very basic non-soap service implementation.
let request = require('request-promise');
let express = require('express');
let app = express();
//Validate the parameters for the request
function validateRequest(req) { ... }
//Transform the request to match the internal API endpoint
function transformRequest(req) { ... }
app.post('/external', function(req, res) {
if(!validateRequest(req))
return res.status(400).json({success: false, error: 'Bad request format');
res.status(200).send();
let callbackUrl = req.query.callback;
let transformedRequest = transformRequest(req);
let internalServiceUrl = 'https://internal.service.com/internal'
request.post(internalServiceUrl, {body: transformedRequest}).then(function (internalResponse){
//Return some of the internal response?
return request.get(callbackUrl, {success: true, processed: true});
}).catch(function (e) {
request.get(callbackUrl, {success: false, error: e});
});
});
Option 2: When I setup the service I want to also setup and listen for incoming requests from the external service whilst also listening for REST requests from the internal service.
There is no "listening" in http. Check socket.io if you need realtime listening. It uses websockets.
Your other option is to poll the internal service (say if you want to check for its availability).
is it possible to reuse a socket.id or use it multiple times?
Let's assume a user views multiple pages of the same site in different browser tabs. I want to use a single socket.id, socket to handle them all.
If a user receives a notification it should popup on all tabs with a single socket.emit.
It is possible
From the dates of previous responses I assume it might not have been possible in previous versions of socket.io, but I can confirm that I'm successfully reusing socket ids on reconnections with socket.io 2.3.0.
You just need to override io.engine.generateId. Whatever that method returns will be the id assigned to the socket. Here are the docs about generateId.
As far as I've experimented myself, there are two situations when that method is called. During connections and reconnections.
The method io.engine.generateId receives the originating request object as the argument, so we can use it to figure out if we want to reuse the id or get a fresh new one.
Example
As an example, I'll show how you would reuse an id sent from the client, or create a new one when the client doesn't send it. The id will be sent on the handshake request as the query param socketId.
1. Override io.engine.generateId
First you need to override io.engine.generateId, which is the method that assigns IDs. On the server you need to do something like this.
const url = require('url')
const base64id = require('base64id')
io.engine.generateId = req => {
const parsedUrl = new url.parse(req.url)
const prevId = parsedUrl.searchParams.get('socketId')
// prevId is either a valid id or an empty string
if (prevId) {
return prevId
}
return base64id.generateId()
}
That way, whenever you send the query param socketId in the handshake request, it will be set as the socket id. If you don't send it you'll generate a new one using base64id. The reason to use that library in particular is because that's what the original method does. Here you can find the source code.
2. Send the information on the connection request
Once you have that, you need to send the socketId param from the client. This is described in the docs.
const socket = io.connect(process.env.WEBSOCKET_URL, {
query: {
socketId: existingSocketId || ''
}
})
process.env.WEBSOCKET_URL would be the URL where your web socket is listening.
Note that this will work when connecting, but you might want to update the query on reconnection.
3. Send the information on the reconnection request
On the same section of the docs it explains how to update the query params before reconnection. You just need to do something like this.
socket.on('reconnect_attempt', () => {
socket.io.opts.query = {
socketId: existingSocketId || ''
}
});
Just like that you'll be reusing the same socket id as long as it is sent from the client.
Security concerns
It's probably a bad idea to trust information sent from the client to assign the socket id. I'd recommend sending a cryptographically signed payload, storing that payload in the client, and send it back to the server when connecting and reconnecting. That way the server can check that the payload can be trusted by verifying the signature.
Using the same example above, we would send something like this to the client, maybe .on('connect'):
{
socketId: 'foo',
signature: SHA_256('foo' + VERY_SECRET_PASSWORD)
}
The client would store that payload and send it back on connecting or reconnecting, in the same way we were sending socketId before.
Once the server receives the signed payload, inside io.engine.generateId we could check that the signature in the payload matches the hash we produce using the ID and the VERY_SECRET_PASSWORD.
You can't reuse Socket.IO connection IDs since they are created during the client-server handshake, but there are alternative methods. I don't have any examples, but you can modify the Socket.IO client to pass along a query string when the handshake is being performed. Then you can tell the server to handle the client based on the query string, and later fetch all client IDs with a certain query string.
Another method you could use would be to use namespaces. Assuming you have some type of session system, you could create a session-specific namespace, and connect clients with that session ID straight to that namespace.
Multiple sites?
No, that's not possible. It would be possible if you open those sites into iframes in your webapp, I guess.
Another option would be to build a browser plugin that opens a socket connection.