I am building chat app.My server code:
socket.on('sendMessage',function(data){
var to=data.To;
var from=data.From;
var message=data.Message;
var sendData={
to:to,
from:from,
message:message,
messageId:randomId
}
io.to(to).emit("processmessage",sendData);
});
It is working well when receiver is online.But when I plugged out the ethernet cable and when someone sends a message to me the message doesn't come after I reconnected. (I am reconnecting within 10 seconds.)
Ps:After I reconnected new messages are coming without any problem.I lost the messages when I reconnecting.
How can I resolve this ?
You can implement a retry system that uses each messages unique id and timestamp to check whether it has reached the other side.
You would require that the receiver send back an acknowledgement that it has received a message of a particular ID.
Set a timeout (T) where you will retry a message after (T) seconds if it hasn't been acknowledged.
You can keep a buffer of messages that you send, then for every message that you confirm has been received (via an acknowledgement from the receiver), you can remove it from the buffer. After receiving an acknowledgement, you can also check the oldest message in your buffer to see if it is older than (T). If it is, you can retry the message and reset the timestamp. Continue checking the next oldest until you find one that is not older than (T). Then move on to sending next message.
This procedure will help your system recover messages that may have not made it to the receiver during downtime in the network.
Related
Intro
We're developing a system to support multiple real-time messages (chat) and updates (activity notifications).
That is, user A can receive via Web Socket messages for :
receiving new chat messages
receiving updates for some activity, for example if someone like their photo.
and more.
We use one single WebSocket connection to send all these different messages to the client.
However, we also support multiple applications/clients to be open by the user at the same time.
(i.e - user A connect on their web browser, and also from their mobile app, at the same time).
Architecture
We have a "Hub" that stores a map of UserId to a list of active websocket sessions.
(user:123 -> listOf(session#1, session#2))
Each client, once websocket connection is established, has its own Consumer which subscribes to a pulsar topic "userId" (e.g - user:123 topic).
If user A connected on both mobile and web, each client has its own Consumer to topic user:A.
When user A sends a new message from session #1 to user B, the flow is :
user makes a REST POST request to send a message.
service stores a new message to DB.
service sends a Pulsar message to topic user:B and user:A.
return 200 status code + created Message response.
Problem
If user A has two sessions open (two clients/websockets), and they send a message from session #1, how can we make sure only session #2 gets the message ?
Since user A has already received the 200 response with the created message in session #1, there's no need to send the message to him again by sending a message to his Consumer.
I'm not sure if it's a Pulsar configuration, or perhaps our architecture is wrong.
how can we make sure only session #2 gets the message ?
I'm going to address this at the app level.
Prepend a unique nonce (e.g. a guid) to each message sent.
Maintain a short list of recently sent nonces,
aging them out so we never have more than, say, half a dozen.
Upon receiving a message,
check to see if we sent it.
That is, check to see if its nonce is in the list.
If so, silently discard it.
Equivalently, name each connection.
You could roll a guid just once when a new websocket is opened.
Or you could incorporate some of the websocket's addressing
bits into the name.
Prepend the connection name to each outbound message.
Discard any received message which has "sender" of "self".
With this de-dup'ing approach
there's still some wasted network bandwidth.
We can quibble about it if you wish.
When the K-th websocket is created,
we could create K topics,
each excluding a different endpoint.
Sounds like more work than it's worth!
I used SSE for push notification. But i can't get an error or a close event in the server when the client's wifi/mobile data unconditionally disconnect without closing the connection in the right way.
It always see the client as online for about 15 minutes before getting connection closed message.
I used regular implementation of SSE in nodejs and express.
Is there any way to check response.write() whether the message delivered to the user or not?
To get a socket error you need the server to attempt to write data; there is no other way to detect when the connection has dropped except to try using it.
As described in chapter 5 of Data Push Apps with HTML5 SSE, what I do is: a) have the server send out a keep-alive message every e.g. 30 seconds. I normally have the message just be the current datestamp, but it could be anything; b) have the client disconnect and reconnect if it hasn't received any messages for 45 seconds.
I am having serious problems to make messages delivery fail proof in a chat system.
Having several node.js and live communication via websocket to the clients, I use rabbit to callback the correct consumer at a specific node.
I declare my queues as {durable: true, prefetch:1, expires: 2*3600*1000, autoDelete: true}
consumerOption is {noAck: false, exclusive: false}
Once I receive a message from the server, I callback the server, get the message, and use message.ack(false)
Sometimes, a message appears with a pendent ACK in rabbit and as I would expect, the consumers stop being callbacked.
Here is my overall strategy:
1- when socket disconnects, I recover the queue using queue.recover() during the the reconnection/connection (more frequent).
2- When I send a message to the server and not receive it back, I send a message to the server to recover the queue.
3- I use the socket callback function to send the ack confirmation. On the server, I use message.ack(false) The server keeps a hashmap {[ackCode: string]: RabbitMessage} and I send the ackCode back to the server, so it can retrieve the correct message and ack it.
5- If client is not receiving any message for 2 minutes, I ask to the server to recover the queue.
The step 5 should not exist but even with this step, sometimes I send a recover queue request to the server, the server executes the command, but nothing happens and chat is freeze.
These are very difficult events to debug. I am using a Typescript library which is 3 year without any commit and this could be one of the causes.
Regarding the strategy, is it correct? Any idea on what I could be facing?
What I've learned and why I think that I couldn't use rabbit to solve the specific problem mentioned in the original post.
The domain: A "chat" where the message order is very important (some are chains) and we must be sure that the message will be delivery if/when the client is online.
The problem: We have several node.js servers, sockets are spread among them. Sockets falls all time, and it is common to a client connection that was in the first server be connected again in another. We don't use cookies, session affinity by IP won't handle the issue.
Limitations: That being said, I can't activate a consumer that is currently activated in another server, so if a customer Queue is tied to server 1 I can't activate it in server 2. And all the messages that need to be sent are tied to this specific queue.
Another limitation is that I don't have an easy way to consume queues, re-queue, to know in advance how much not ack messages I have in the queue, aggregate them and bulk send them via socket.
The solution: I am no longer using {noAck: false} and I am controlling the ack in a Redis queue. Thus, I am using Rabbit as a pub-sub, to callback the correct consumers to send the message using the socket. Rabbit wake me up, first thing I do is to put the message at the end of a redis queue. When I send a message via socket, I always start sending the messages from the beginning of the queue, regardless of the message that have just woke me up. I send the message, wait for the callback event, If it is not ok, I re-queue the messages,
After decoupling the pub-sub from the queue/ack control, I can now easily change my rabbit pub/sub from one server to another (declaring using socket.id and no more with the client queue), with no concern of loosing any message. Also, now I am capable of much more advanced operations on my queue.
As my use case don't allow me to use the full power of exchanges/binds (i have complex routing rules), I am evaluating the possibility of changing from rabbit to redis pub/sub, but in this case, I would continue to differentiate pub/sub from the queue.
After more than a month trying to make rabbit working in this scenery, I think that I was using a good technology to the wrong use case. It is much simpler now.
I am using socket.io to send data from my sever to clients. There are situations when a client looses its connection but the server gets to know about this only when the next heartbeat is not recieved from the ckient.
the messages that are sent between the client loosing its network connection to the time when the sever derives this from absense of heartbeats are lost and I am not able resend them when the client rconnects.
I know there I can send a callback in my message which the client will call on successfull delivey of message. however this callback is asynchronous and I Am not aware of any way by which I can getto know that the message delivery failed. Can anyone please help me findhow can I capture a failure to delive a message.
Thanks in advance
According to the documentation, you can configure "max reconnection attempts" for
How many times should Socket.IO attempt to reconnect with the server after a a dropped connection. After this we will emit the reconnect_failed event.
I have minimum 3 TCP client, each has a Thread. I'm sending out messages and waiting for the answer, but sometimes I have to wait to receive the response from all client, this is depending what kind of message sent the server out. I already made to send messages to the clients and receiving, but when I have to wait for the other client response I couldn't do that until now.
As far as you didn't mention your environment/language, I assume C#/.NET 4
You need a mechanism for each client to signal the arrival of a response. This is usually done with AutoResetEvents: Each client sends his response back to the server. The server itself can extract from the reponse (or any other property, e.g. the connection) with client has sent it. Then he sets the apporpriate AutoResetEvent.
The thread that formerly initiated sending the message can afterwards wait for all AutoResetEvents to be set.