I am new to Node and Redis, so forgive me if I'm missing something crucial here. I am trying to store a Node.js net.Socket object in Redis, using the node_redis client, so I could reuse a connection made previously on this Socket.
I am storing the socket as a Value as a part of a "users" set:
client.sadd("users", username); //username is a string
client.hmset(username, "ip", socket.remoteAddress, "connection", net.Socket(socket));
At a different point, I am retrieving this socket as:
client.smembers("users", function(err, replies) {
replies.forEach(function(reply,i){
if(recipient == reply) { //recipient is the username i've got
client.hget(reply, "connection", function(err, obj) {
console.log("socket's remoteaddress = " + net.Socket(obj).remoteAddress);
net.Socket(obj).write("asdasdasd");
});
}
});
});
But, the:
net.Socket(obj).remoteAddress); is logged as 'undefined'.
Also,
net.Socket(obj).write("asdasdasd"); gives me and error saying:
Error: This socket is closed.
So I guess my question is - can you store Sockets this way in Redis and expect them to work upon retrieval? Is there a correct / better way to do this?
P.S. I tried retrieving the Socket without the cast to net.Socket and it still didn't do any good.
Sockets and socket APIs (such as Berkeley Sockets) intrinsically handle special system-specific resources for which it just wouldn't make any sense to store or reference on a remote system, or even serialize for that matter.
Although, in reality, sockets are often uniquely identified by file descriptors so you could easily store that small, unique integer in a database for later access; however, that integer would only be meaningful during the active lifetime of that socket within that process instance. A strategy such as this one will be fraught with challenges, mainly having to do with the transient nature of the socket file descriptor number.
as #maerics said , redis store only string. so your socket data will be destroyed and will give undefined as answer.
you can store your socket like this.
var OBJECT = Object.prototype;
OBJECT.rhash = {};
OBJECT.rset = function(id, object) {
OBJECT.rhash[id] = object;
return id;
};
OBJECT.rget = function(id) {
return OBJECT.rhash[id];
};
this will store reference to that object and reuse it again..
var id = OBJECT.rset("a123",socket);
var ref = OBJECT.rget("a123");
console.log(ref.remoteAddress);
Regarding the IP address, it's on a different hash key (not connection, but ip). So to get the value you just read from that key:
client.hget(reply, "ip", function(err, obj) {
console.log("socket's remoteaddress = " + obj);
});
Related
I am building a web socket application using node and i want to store my currently connected users ,so i saved it as a user Id key and socket Id value but when the user disconnect i only knows it's socket Id so i will not be able to remove it from Redis (since Redis only allow search by key) so how would i solve it ? i tried to reverse the key and value but it's impossible as a socket id is not unique
const isUserExist = async (socketId) => {
return await redisClient.get(socketId);
};
const addUser = async (socketId, userId) => {
if (await isUserExist(socketId)) return console.log("user already exist");
await redisClient.set(socketId, userId);
console.log("user is set inside database ");
};```
i have actually discovered that socket object carries the socket id of the current connected user , so i used it and my problem was solved and we also need to store the id as the key and the socket id as the value
I know that this question was asked already, but it seems that some more things have to be clarified. :)
Database is designed in the way that each user has proper privileges to read documents, so the connection pool needs to have a connection with different users, which is out of connection pool concept. Because of the optimization and the performance I need to call so-called "user preparation" which includes setting session variables, calculating and caching values in a cache, etc, and after then execute queries.
For now, I have two solutions. In the first solution, I first check that everything is prepared for the user and then execute one or more queries. In case it is not prepared then I need to call "user preparation", and then execute query or queries. With this solution, I lose a lot of performance because every time I have to do the checking and so I've decided for another solution.
The second solution includes "database pool" where each pool is for one user. Only at the first connection useCount === 0 (I do not use {direct: true}) I call "user preparation" (it is stored procedure that sets some session variables and prepares cache) and then execute sql queries.
User preparation I’ve done in the connect event within the initOptions parameter for initializing the pgPromise. I used the pg-promise-demo so I do not need to explain the rest of the code.
The code for pgp initialization with the wrapper of database pooling looks like this:
import * as promise from "bluebird";
import pgPromise from "pg-promise";
import { IDatabase, IMain, IOptions } from "pg-promise";
import { IExtensions, ProductsRepository, UsersRepository, Session, getUserFromJWT } from "../db/repos";
import { dbConfig } from "../server/config";
// pg-promise initialization options:
export const initOptions: IOptions<IExtensions> = {
promiseLib: promise,
async connect(client: any, dc: any, useCount: number) {
if (useCount === 0) {
try {
await client.query(pgp.as.format("select prepareUser($1)", [getUserFromJWT(session.JWT)]));
} catch(error) {
console.error(error);
}
}
},
extend(obj: IExtensions, dc: any) {
obj.users = new UsersRepository(obj);
obj.products = new ProductsRepository(obj);
}
};
type DB = IDatabase<IExtensions>&IExtensions;
const pgp: IMain = pgPromise(initOptions);
class DBPool {
private pool = new Map();
public get = (ct: any): DB => {
const checkConfig = {...dbConfig, ...ct};
const {host, port, database, user} = checkConfig;
const dbKey = JSON.stringify({host, port, database, user})
let db: DB = this.pool.get(dbKey) as DB;
if (!db) {
// const pgp: IMain = pgPromise(initOptions);
db = pgp(checkConfig) as DB;
this.pool.set(dbKey, db);
}
return db;
}
}
export const dbPool = new DBPool();
import diagnostics = require("./diagnostics");
diagnostics.init(initOptions);
And web api looks like:
GET("/api/getuser/:id", (req: Request) => {
const user = getUserFromJWT(session.JWT);
const db = dbPool.get({ user });
return db.users.findById(req.params.id);
});
I'm interested in whether the source code correctly instantiates pgp or should be instantiated within the if block inside get method (the line is commented)?
I've seen that pg-promise uses DatabasePool singleton exported from dbPool.js which is similar to my DBPool class, but with the purpose of giving “WARNING: Creating a duplicate database object for the same connection”. Is it possible to use DatabasePool singleton instead of my dbPool singleton?
It seems to me that dbContext (the second parameter in pgp initialization) can solve my problem, but only if it could be forwarded as a function, not as a value or object. Am I wrong or can dbContext be dynamic when accessing a database object?
I wonder if there is a third (better) solution? Or any other suggestion.
If you are troubled by this warning:
WARNING: Creating a duplicate database object for the same connection
but your intent is to maintain a separate pool per user, you can indicate so by providing any unique parameter for the connection. For example, you can include custom property with the user name:
const cn = {
database: 'my-db',
port: 12345,
user: 'my-login-user',
password: 'my-login-password'
....
my_dynamic_user: 'john-doe'
}
This will be enough for the library to see that there is something unique in your connection, which doesn't match the other connections, and so it won't produce that warning.
This will work for connection strings as well.
Please note that what you are trying to achieve can only work well when the total number of connections well exceeds the number of users. For example, if you can use up to 100 connections, with up to 10 users. Then you can allocate 10 pools, each with up to 10 connections in it. Otherwise, scalability of your system will suffer, as total number of connections is a very limited resource, you would typically never go beyond 100 connections, as it creates excessive load on the CPU running so many physical connections concurrently. That's why sharing a single connection pool scales much better.
I'm trying to make a game, which works on rooms, lobby and such (imagine the chat app, except with additional checks/information storing).
Let's say, I have a module room.js
var EventEmitter = require('events');
class Room extends EventEmitter {
constructor (id, name) {
super();
this.id = id;
this.name = name;
this.users = [];
}
}
Room.prototype.addUser = function (user) {
if(this.users.indexOf(user) === -1) {
this.users.push(user);
this.emit('user_joined', user);
} else {
/* error handling */
}
};
module.exports = {
Room: Room,
byId: function (id) {
// where should I look up?
}
};
How can I get exactly this object (with events)? How can I access events emitted by this object?
In a single instance of node, I would do something like:
var rooms = [];
var room = new Room(1234, 'test room');
room.on('user_joined', console.log);
rooms.push(room);
Also, I don't quite understood how Redis is actually helping (is it replacement of EventEmitter?)
Regards.
EDIT: Would accept PM2 solutions too.
Instead of handling rooms in Node, you can replace them with channels in Redis).
When a new client wants to join in a room, the NodeJS app returns it the ID of this given room (that is to say the name of the channel), then the client suscribes to the selected room (your client is directly connected to Redis.
You can use a Redis Set to manage the list of rooms.
In this scenario, you don't need any event emitter, and your node servers are stateless.
Otherwise, it would mean Redis would be exposed on the Internet (assuming your game is public), so you must activate Redis authentication. A major problem with this solution is that you have to give the server password to all clients, so it's definitely unsecure.
Moreover, Redis' performances allow brute force attacks so exposing it on Internet is not recommended. That's why I think all communications should go through a Node instance, even if Redis is used as a backend.
To solve this, you can use socket.io to open sockets between Node and your clients, and make the Node instances (not the client) subscribe to the Redis channel. When a message is published by Redis, send it to the client through the socket. And add a layer of authentication to ensure only valid clients connect to a given channel.
Event emitter is not required. It's the Redis client which will be an event emitter (like in this example based on ioRedis)
I am adding the username to the socket object like this which is working fine
socket.on('add user', function (username) {
socket.username = username;
});
Lets assume the username is khawer and now i want to emit to this socket where username is khawer but i am unable to do so.
I have tried this
io.sockets.connected[socket.username].emit('chat message', msg);
And this
io.sockets.sockets[socket.username].emit('chat message', msg);
But both did not work. What am i doing wrong here?
Just assigning a username property to a socket does not make it so that it's indexed by name - thus you cannot do either of the types of lookups you're doing.
If you want to find a socket by username, you will either have to do a brute force search of all the sockets to find the one that has the same user name or you will have to create your own index of sockets by name.
If you want to do a brute force lookup to find it, you could do this:
var list = io.sockets.sockets;
for (var i = 0; i < list.length; i++) {
if (list[i].username === "khawer") {
list[i].emit('chat message', msg);
}
}
You could also put each user into a chatroom with a name that matches their username. Then, you could send to any given username by simply sending to the chatroom by that name. You'd be using the chatroom feature as an index by username. It would just require one extra step to put a socket into a chatroom that matches their username when they connect.
Or, each time a socket connects and disconnects, you could maintain your own socket index by username (this is relatively common).
I'm designing a chat script which I test on my machine using different browsers. I'm tryng to send messages to specific users with socket.io, so here it is :
client:
socket.on('msgFromServer', function (data) {
message = data['message'],
from = data['from'],
to = data['to'];
if($('#chatbox.'+from).dialog("isOpen") === true){
$('#chatbox.'+from+' #messageOutput textarea.readOnly').text(message);
}
else if(($('#chatbox.'+from).dialog("isOpen") !== true)){
createChatbox(from,to,message);
}
});
server:
var users = {};
io.sockets.on('connection', function (socket) {
if( ( users.hasOwnProperty(req.session.name) === false))
users[req.session.name] = socket;
socket.on('msgToServer', function (data) {
for (var u in users){
console.log("%s | %s",u,users[u]);
}
});
});
Well, I'll talk about the structure of code related to the server. It is in charge of storing a user on a 'connection' event. The problem starts when I reload the page: it stores the user from browser A in the users object, if I reload and reconnect stores it again , but when I ask which are the contents of the users object in browser B ... the info is outdated and does not show the same result as when I ask which are the contents of the object in broser A, even though I'm trying to do some cheking of nullity to store vals if users is empty --> if( ( users.hasOwnProperty(req.session.name) === false)). Basically, what I need is a means of storing each socket resource in a container(in fact, doesn't necessarily needs to be an object) with an identifier(req.session.name) and to have such container available to all sessions in all browsers, so when server receives a message from browser A to browser B it could identify it and emit a response to browser B.
I got an I idea of what I wanted from https://github.com/generalhenry/specificUser/blob/master/app.js and http://chrissilich.com/blog/socket-io-0-7-sending-messages-to-individual-clients/
If you look carefully at the code... in chrissilich.com , the author states that we need to store the 'socket.id' (users[incoming.phonenumber] = socket.id), whereas in git generalhenry states we have to store the 'socket'(users[myName] = socket) resource. The latter is the correct one , because the values of socket.id tend to be the same in both browsers... and that value changes automatically , I don't know why is there... I suppose in earlier versions of node it worked that way.
The problem is that socket.id identifies sockets, not users, so if an user has several tabs opened at same time, every tab would have different socket.id, so if you store only one socket.id for an user, every time you assign it, you overwrite previous socketid.
So, beside other possible problems, at least you need to do this or it won't work. I bet that you say about 1 socket for all browsers is that you overwrite the id every time (it happened to me when I started using Socket.IO)
As a general rule, remember that you manage CONNECTIONS and not USERS... an user can have more than one connection!.
On connection
function onConnection( socket ) {
var arr = users[incoming.phonenumber] || null;
if( !arr )
users[incoming.phonenumber] = arr = [];
if( arr.indexOf( socket.id ) === -1 )
arr.push( socket.id ); // Assigns socket id to user
}
On disconnection
function onDisconnect( socket ) {
var arr = users[incoming.phonenumber] || null;
if( !arr ) return; // Should not happen since an user must connect before being disconnected
var index = arr.indexOf( socket.id );
if( index !== -1 )
arr.splice( index, 1 ); // Removes socket id from user
}