I'm working on a private chat messenger application using react-native node and socket.io and need to set up the structure of the project and the data. the internet seems to be overloaded with ton of different answers and I was hoping to get some answers about the problems I have with these solutions
So as far as I understand from the official socket.io documentation
https://socket.io/docs/rooms-and-namespaces/#Namespaces
namespaces were meant to "separate concerns within your application by introducing separation between communication channels" which sounds exactly like what I need for creating a separation between different conversation.
and rooms "Within each namespace, you can also define arbitrary channels that sockets can join and leave."
Here is the data structure I had in mind:
Conversations (namespaces) table -
id, namespace, user1, user2
Messages table-
conversationId, content, from(user1/2)
Here is the general flow I had in mind :
Step 1 -
User connect the main namespace ('/')
Step 2 -
The server returns a list of all user's conversations (namespaces) from the conversation table in the database
Step 3 -
Connect the user to (all?) his namespaces (or should I only connect him to the conversation he enters?)
Step 4 -
Join to default 'chat' room under each namespace (is this really necessary in a private chat application ? I know on slack for example it make sense because we speak to the same group of people under different channels, but is there any other advantage for room that I'm missing? and if it does necessary should the user join all the rooms belongs to him when the applications first fire or should I only join him into the room he enters?)
Step 5 -
When a user enters specific conversations he get all the chat history under this conversation (from Messages table in the database)
Step 6 -
When a new conversation is added we add a record to the conversation table (and we emit 'newConversation' event to the entire namespace/room that include just the sender for now (how do I make sure to notify the receiver about the new namespace related to him should I emit the event to the main name space and validate the access right on connection to namespace attempt or should I use a solution like #)) perhaps this is out of the socket.io scope and should be left to the server to deal with ?
Step 7 -
When a new message is sent we add it to the messages table and emit 'newMessage' event to the entire namespace/room
# on many places I saw some versions of the following solution:
How to send a message to a particular client with socket.io
which basically suggest storing a dictionary relating between each user and his socket id, what I don't understand is, every time a user refresh/login/logout he receive a new socket id and all the events we emitted to his previous socket id will now be lost
My expected result is a chat like whatsapp or facebook messenger (but with private conversations only) where you get notified in messages in other conversation even when you are on the home page and not inside a specific conversation
Does creating a presence channel automatically show the presence info in the same named non-presence channel e.g.
"presence-chat" shows members of room "chat"
OR
Are these two channels completely un-associated and any associations between them must be maintained manually by the app? And the presence channel is just a type of channel that can be arbitrarily named anything with the prefix "presence-"
So if it is the latter, then in a chat room use case one should subscribe the user to both the chat room channel as well as the presence of the chat room channel, which may or may not be named similarly?
Does creating a presence channel automatically show the presence info in the same named non-presence channel e.g.
"presence-chat" shows members of room "chat"
No, chat is a public channel and doesn't not any presence type data or functionality.
Are these two channels completely un-associated
Correct.
and any associations between them must be maintained manually by the app?
Yes. But I'm not sure why you would want to use both a chat and presence-chat channel. All functionality available on a public channel (chat) is also available on a presence channel (presence-chat).
And the presence channel is just a type of channel that can be arbitrarily named anything with the prefix "presence-"
Correct.
Information on channel types can be found via:
https://pusher.com/docs/client_api_guide/client_channels#channel_types
So if it is the latter, then in a chat room use case one should subscribe the user to both the chat room channel as well as the presence of the chat room channel, which may or may not be named similarly?
As above, presence channels offer all functionality of a public channel. So there's no need to use two channels where one can do everything.
Please let me know if I've missed a reason why you need two use two channel types. If so, I'll update my answer.
I am using PubNub to create a Chat. Each chatroom is a PubNub channel. The challenge now is: how do I sort my channels such that channels with most recent posts should be on top.
I can think on 2 possibilities:
Server will listen to all channels, when a message is received, it logs it. So a chatroom model might look like {id, name, users, lastUpdate}
Everytime message is posted, app will also call server to pass in that message
Both methods doesnt seem to correct? Is there a better way? The first will require the server to listen on all channels. The second will require server to handle a request for each message.
How about using timestamp?
Actually, PubNub offers Presence APIs that lets you monitor the state of each channel, with timestamp (which specified as 17-digit precision unix time).
http://www.pubnub.com/knowledge-base/discussion/276/presence
I have 40 categories and each category has 10-100 subcategories. By default, the user listens to all categories and subcategories. I want to give each user the ablity to select to unbind from the whole category or from a specific subcategory. So, right now what I have is each category is a channel, and each subcategory is a event.
Now, I have something like each user is bound to 2000-3000 events, and I know this is wrong, so what is the right way to let the user to filter between 3000 events? Is it okay to bind to that many events?
It's important to remember that when you subscribe to a channel all events for the channel will be sent to the client (Pusher - clients), even if you have not bound to the event.
With the information above in mind, I'd recommend using channels to filter data. The overhead when subscribing to a channels isn't great. For example, subscribing to 40 channels wouldn't represent any significant resource usage. You will need to consider if the channels are public (anybody can subscribe) or private where each call to pusher.subscribe( 'private-channel-x' ); will result in an authentication request to your server. There is a multi-auth plugin that allows batching of authentication requests to take place.
One solution is to have each user subscribe to their own notification channel and send them events for the things that they are interested in on that single channel. You can use the multi publish functionality for this, which lets you send the same event on many channels. This may be useful if you wish to send the same event to multiple users. However, this solution may not be as elegant from an information architecture point of view.
The best solution here really depends on your application. But with the above you now have all the facts which will let you make the most efficient choice.
I'm using Php and MySQL.
I have just signup for pubnub push API and successfully made my first push notification using the PHP Push API provided by Pubnub. I'm new into this Realtime technology so I have come up with some questions which I find confusing to understand myself. I have googled many times and searched all over stackoverflow. I didn't get any relevant suggestions or questions elsewhere, so I'm writing down my question here seeking your advice and expertise help.
Pubnub says it's not a good thing to create more than two channels per client. So, in my application, I have a requirement to create more than two channels to listen to notifications happening everywhere on my website but I will go with two channels for every logged in users as Pubnub suggested.
Logged in users listens to Channel1-Public
Logged in users listens to private UsersOwnDynamic-Channel to receive notifications related and only meant for him.
FYI: This link in PubNub says about creating LongChannel names to avoid Channel Snooping
My questions are given below:
A. Do I always have to create a new private Dynamic channel name everytime I logged into the website. If so, how would other users know how to send a notification to my private Channel.Or, do I just need to have only one static channel name stored in database table, so that other authenticated users will query the table and get my private channel name to sent me notifications. If this is the case, don't you think if a hacker get hold of some private channel name of certain users, they will be able to listen to that channel?
B.I'm using PHP and MySQL, so I still cannot think out a way or come up with a solution to send message to private channels of another user.
Lets take an example for a simple friend request system.
- UserA sends a friend request to UserB.
- UserB is listening to his own dynamic private channel name called DynamicPrivateChannelB
(how will UserA find the channel name for UserB which is private? Im thinking the only way for this is that the private channel of UserB should be stored in a database table for every loggedin users to query. Am I thinking the right way ? )
<?php
//first way. How can i possibly achieve this.
$sqlquery = "sent friend request from userA to userB";
require('Pubnub.php');
$pubnub = new Pubnub( 'pubkey', 'subkey' );
$pubnub->publish( array(
'channel' => 'how do i find the private channel name for userB to sent this notification?',
'message' => array('friend_request' => 'A friend request') )
);
//2nd way ? Is this the right way ?
$sqlquery = "sent friend request from userA to userB";
$privatechannelofuserB = "get the channel name of userB from the db table";
require('Pubnub.php');
$pubnub = new Pubnub( 'pubkey', 'subkey' );
$pubnub->publish( array(
'channel' => '$privatechannelofuserB',
'message' => array('friend_request' => 'A friend request') )
);
?>
C. If we are always generating dynamic private channel names, storing in database table, updating whenever new dynamic channel names are generated. I think it will cause a problem because some message won't get delivered as new dynamic private channel names replaces old ones.
D. So, I have many notifications to sent to a single channel like New Friends request, New Private message replies, New gifts requests and many others like that. How do I sent all these data to a channel and how to find out and parse the incoming new notifications data. I know JSON is the format to send but im not sure about the format of sending.
According to this link, a single Pubnub channel can contain only up to 100 messages. Does it mean that if 200 messages comes at once to a single channel the first 100 gets delivered and the remaining is on queue ? How about if 10,000 messages comes at once to a single channel ? Do all the remaining messages stays in queue ? if so, how does it gets delivered to the subscriber in realtime ?
Let me give another simple scenario that I'm trying to achieve.
UserA is authenticated and logged in to the website.
UserA generates his own Dynamic channel name, UserAx732dsw3efsdfsdfsdf
UserA starts listening to his newly created channel UserAx732dsw3efsdfsdfsdf (Now, userA should start receiving messages from others)
- UserB sends a private message to userA. (Now, only userA should get notified on his private channel about the new private
message, how can userB or the System find out the channel
name UserAx732dsw3efsdfsdfsdf because, this is a private channel
generated dynamically by userA , neither the System or
userB has accessed to it. This same thing is happening for userB as well, if userB should be notified by any other entity or by the system again, there should be a way to find out the dynamic
channel name of userB .
Another issue is this scenario is that, if a user is dynamically generating channel name everytime he/she logged in to the website. What will happen to all the messages that was sent to the dynamic channel ? Do pubnub saves all the created channel names on their server? Is there any way that a system or a user can find out whether a channel name is still being own and atleast one user is listening to the channel ?.
I'm curious to know this because of the following concepts that I have:
UserA creates dynamicChannelA when he/she logged in to the website at 1AM.
UserA starts getting lots of notification pushout to his dynamic channel dynamicChannelA
Now, UserA logs out from the website at 1:30 AM, what will happen to the many other users who are still pushing out notification
to his dynamicChannelA because by the next time when UserA
logs into the website, UserA will be listening to different
dynamic channel name.UserA won't be listening to his previous
channel dynamicChannelA .
I'm thinking to use the method of retrieving channel name of a particular user from the database table. Is there any method or way to prevent unauthorised subscription of the channel? Because anybody can subscribe to the channel name if they have the subscribe key and channel name no matter how long the channel name is. I'm just curious because all subscription is happening in the client side and the subscription key and channel names are visible.
There's no one single way of tackling the issues that you've run into. Our customers have used a wide variety of design patterns to deal with them. I myself have run into this type of thing building PubNub apps and I'll help you as much as I can.
Pubnub says its not a good thing to create more than two channels per client. So, in my application, i have a requirement to create more than two channels to listen to notifications happening everywhere on my website but i will go with two channels for every logged in users as Pubnub suggested.
Logged in users listens to Channel1-Public
Logged in users listens to private UsersOwnDynamic-Channel to receive notifications related and only meant for him.
This is a good way to do it, and the way many of our massive scale customers do it. A global channel and a private, user-only channel.
A.
Do i always have to create a new private Dynamic channel name everytime i logged into the website.
Not necessarily, though this is a good way. You can use PUBNUB.uuid() in JavaScript on the client-side to do this. Or, generate it server-side using PHP and render it to the client. Maybe you could set it as a cookie so the client always has access to it.
If so, how would other users know how to send a notification to my private Channel.
They could get the id's from the PHP server; either via the global channel or the user's own private channel, which they are listening to.
Or, do i just need to have only one static channel name stored in database table, so that other authenticated users will query the table and get my private channel name to sent me notifications.
You could do it this way too. You might have the global channel the users can send to be different then the global channel they are listening to. Only the server has that subscribe key. Thus, authenticated users send a message to the server that tells it "I need the appropriate user keys", then the server does a query and sends a message back on that users private channel.
If this is the case, don't you think if a hacker get hold of some private channel name of certain users, they will be able to listen to that channel?
If you withhold the subscribe key on the global send channel, only the server can see the chatter on that channel.
B.
Im using PHP and mysql, so i still cannot think out a way or come up with a solution to send message to private channels of another user.
Lets take an example for a simple friend request system.
- UserA sends a friend request to UserB.
- UserB is listening to his own dynamic private channel name called DynamicPrivateChannelB
(how will UserA find the channel name for UserB which is private? Im thinking the only way for this is that the private channel of UserB should be stored in a database table for every loggedin users to query. Am i thinking the right way ? )
This is very similar to your previous question. There's no one way to do it, but the design pattern I outlined above should work. To recap this design pattern:
Server side
Listens on Global-user-send-channel for user messages. The server is the only entity that has this subscribe key
Can query db to get user-ids, and then send to the various ids at will
Can also send on the Global-user-receive-channel, which all clients are listening on. The server is the only entity that has this publish key.
Client side
Listens on Global-user-receive-channel. This is how it gets mass server broadcasts. Cannot send on this channel (only has subscribe key)
Sends server messages on Global-user-send-channel. Cannot recieve on this channel (only has publish key)
Listens on private user channel. This is how user gets private messages. It can also use this for client-to-client communication.
Prevent abuse by appending all private messages with a private, per-user key stored on the server, and provided during the initial page-load. That way, a client knows if a message claiming to be from the server is legit.
C.
If we are always generating dynamic private channel names, storing in database table, updating whenever new dynamic channel names are generated. I think it will cause a problem because some message won't get delivered as new dynamic private channel names replaces old ones.
If you're careful about when you generate new channel names, this shouldn't be a problem. Keep in mind, the client can always say on the Global-user-send-channel 'hey, I'm here! this is my id. keep me updated'. I usually design my apps to have the clients automatically shout this out every 30 seconds or so.
D.
So, i have many notifications to sent to a single channel like New Friends request, New Private message replies, New gifts requests and many others like that. How do i sent all these data to a channel and how to find out and parse the incoming new notifications data. I know JSON is the format to send but im not sure about the format of sending.
JSON is good for sending and recieving. The way I do it is to have a property called "name" which defines what type of message it is. For example:
{
"id" : "blah_blah_unique_id", // sender_client_id
"name" : "friend_request", // type of message
"data" : { // the data itself
"requested_friend_id" : "blah_blah_some_other_unique_id"
}
}
You can actually use whatever format you want, but we'll wrap it in JSON (usually that means just wrapped in quotes) when it gets pushed through PubNub.
Hope this helps!
New Questions
According to this link, a single Pubnub channel can contain only upto 100 messages. Does it mean that if 200 messages comes at once to a single channel the first 100 gets delivered and the remaining is on queue ? How about if 10,000 messages comes at once to a single channel ? Do all the remaining messages stays in queue ? if so, how does it gets delivered to the subscriber in realtime ?
The 100 message limit is in regards to PubNub.history. If someone is subscribed and 200 messages come in, they will receive all 200 messages.
(Now, only userA should get notified on his private channel about the new private message, how can userB or the System find out the channel name UserAx732dsw3efsdfsdfsdf because, this is a private channel generated dynamically by userA , neither the System or userB has accessed to it. This same thing is happening for userB as well, if userB should be notified by any other entity or by the system again, there should be a way to find out the dynamic channel name of userB .
There's no one-size-fits-all solution to this question, but what I'd do is have the server generate that unique id on page load and render it to the client on your initial HTTP request.
Another issue is this scenario is that, if a user is dynamically generating channel name everytime he/she logged in to the website. What will happen to all the messages that was sent to the dynamic channel ? Do pubnub saves all the created channel names on their server?
You don't have to dynamically generate every time. You could.... but you also set a cookie with that unique id, or pull it from the database and render it to the client on page load (that's what I'd do). We don't save channel names.
Is there any way that a system or a user can find out whether a channel name is still being own and atleast one user is listening to the channel ?.
Not out of the box, but you could implement this easily. Just have your server send out a ping and set up your clients to always respond to pings if they're listening.
Now, UserA logs out from the website at 1:30 AM, what will happen to the many other users who are still pushing out notification to his dynamicChannelA because by the next time when UserA logs into the website, UserA will be listening to different dynamic channel name.UserA won't be listening to his previous channel dynamicChannelA .
The way you can prevent this is periodic (every 30 seconds?) pings from the server, who can keep track if users are still there. In the coming months we'll be launching a presence API to do this automatically, btw.
Im thinking to use the method of retrieving channel name of a particular user from the database table. Is there any method or way to prevent unauthorised subscription of the channel? Because anybody can subscribe to the channel name if they have the subscribe key and channel name no matter how long the channel name is. Im just curious because all subscription is happening in the client side and the subscription key and channel names are visible
The main way is strategically witholding publish/subscribe keys. You're right that anyone with the proper details can listen in - this is a big problem of client-only systems. For the time being, you'll have to come up with creative ways to get around it.