Determine when http end point stopped sending POST request - node.js

I have an IoT (Arduino) device that sends HTTP POST request every 6 seconds to my NodeJs server.
The data from the device is sent in JSON format.
At some point, the device stops sending data (let's say the GSM module got turned off or maybe the device got turned off for some reason) and I need to determine when that happens and output something like: 'device status: offline'
I use ExpressJs to handle the server functionality and I can easily handle any POST/GET request from/to the IoT device but how to determine when it's no longer sending POST requests??
tried to check if the IP address of the IoT device (it uses GSM module with a SIM card to send data to my NodeJS server) is reachable but I'm new to Node and I use body-parser so the request is automatically parsed to JSON and I'm not sure how to determine if the IP is reachable.
setDeviceStatus() is a function that updates the device status (online/offline) in a cache file/table in my database
app.post('/floraData', (req,res,next) => {
var jsonObj = req.body;
var deviceStatusTimer = setInterval(function(){
if(jsonObj == null){
setDeviceStatus(0);
console.log("device offline");
}
else{
setDeviceStatus(1);
console.log("device is online");
}
}, 10000);
next();
});

WebSocket will be an easier approach. As #Gonzalo.- pointed out in the comment, you can send ping request from the server to health check the IoT device.
However, WebSocket has its limitation that it can hardly scale out if you have a lot of IoT devices. If so, building a stateless application, like what you are doing now, will be more suitable in long run.
Answering your question, instead of using the IP (multiple devices can share the same IP) to health check the IoT device, the device can send the device_id (e.g. MAC address, custom UUID, etc.) in the POST request. Whenever the server receives the POST request, it should store the device_id in Redis, for example, with the last_online timestamp.
Depending on your use case, you can build another application (e.g. lambda + cloudwatch) to check the Redis periodically for any devices which the last_online time was X minutes ago.

Related

How to prevent socket io client (browser) from being overflowed with huge payload coming from server?

I have a React.js client app and a Node.js server app and the Node.js app receives json data in real time via socket.io from another microservice. The JSON data is sent very often and this breaks the client app. For example:
I stop the server but the client still receives data
If I try refreshing the browser, it takes a lot of time to refresh
It also used to disconnect and reconnect the sockets (I fixed this by increasing the pingTimeout but that did not solve the other problems)
I also used maxHttpBufferSize and updateTimeout by increasing them but that does not really help. Decreasing the maxHttpBufferSize stops the messages from being received but I want them to be received just in a manner which does not break my client application.
Any advices on what I can do to improve my situation?
EDIT:
It could also work if I do not send all messages but skip every second or so but I am not sure how to achieve this?
Backpressure can be implemented with acknowledgements:
the client notifies the server when it has successfully handled the packet
socket.on("my-event", (data, cb) => {
// do something with the data
// then call the callback
cb();
});
the server must wait for the acknowledgement before sending more packets
io.on("connection", (socket) => {
socket.emit("my-event", data, () => {
// the client has acknowledged the packet, we can continue
});
})
Reference: https://socket.io/docs/v4/emitting-events/#acknowledgements
Note: using volatile packets won't work here, because the amount of data buffered on the server is not taken in account
Reference: https://github.com/websockets/ws/blob/master/doc/ws.md#websocketbufferedamount

NodeJS WebSockets routing messages from devices to Angular gui through NodeJS

I've created a NodeJS application that implements a socket manager like so:
import { Server } from "ws";
import clientObj from "../objects/clientObj";
import serverObj from "../objects/serverObj";
class socketMgr {
serverA = undefined;
serverB = undefined;
confA = undefined;
confB = undefined;
init(cA, cB) {
this.confA = cA;
this.confB = cB;
}
serve()
{
// server between devices and nodejs
if (this.confA.host && this.confA.port)
{
this.serverA = new Server({ host: this.confA.host, port: this.confA.port });
this.serverA.on("connection", connection => new clientObj(this.confA, conn));
}
// server between nodejs and gui
if (this.confB.host && this.confB.port)
{
this.serverB = new Server({ host: this.confB.host, port: this.confB.port });
this.serverB.on("connection", conn => new serverObj(this.confB, conn));
}
return true;
}
}
export default socketMgr;
This manager declares and initializes two socket servers that send and receive messages. The whole flow goes like this (simplified on a 1 to 1 approach):
device (clients written in native C++ or other, no GUI) <==> NodeJS client/server <==> gui client (devices running Win, iOS or Unix based OS)
device connects to NodeJS and creates a client (that implements event listeners) on serverA
client on serverA replies to the handshake request from the device
when handshake is successfull, device starts emmiting messages periodically to NodeJS (client on serverA)
NodeJS client processes messages and saves some states to the DB (sql), not entire messages because that would be an absolute overkill
TODO:
Once a message is processed on the serverA client, that message has to be emmitted somehow to the corressponding gui client (hosted on a different port).
This is the fun part where web sockets serverB comes into place. I don't really know if this is the right approach but I'd like to use ServerB instance to re-route/broadcast to the gui. Many devices (hundreds or even thousands) will eventually send messages like this and many users can watch these messages from the gui side (monitoring).
Can re-routing/broadcasting messages to the gui be achieved without the serverB instance?
How can I achieve this type of re-routing/broadcasting of messages in NodeJS?
EDIT: This issue is a lot more complicated than my first estimation showed. Devices have to communicate through two web socket servers and connections must be managed seperatelly because these devices on both sides aren't equal in any way. Routing between them must be handled programmatically.
If I can provide any additional info. on the subject, please let me know.
Thank you for looking into this.
Best Regards.
You dont need to create two server for this. Device and GUI both is client. When device emits data to server you can send this data to GUI by allowing subscribing GUI client to device messages, by topic name or by device ID.

Node js reuse response object

First, a little background:
I'm trying to write a mobile phone app that receives data from a nodejs server at an irregular interval. Since this is a mobile app, I would prefer not to poll the server or use a websocket, as I believe that would drain the battery. Instead, I'd like the server to keep track of client IP addresses and push data.
My plan is to have the app send a simple message to the server whenever the phone's connection status changes (i.e, if the phone goes from a wifi connection to mobile data one, send a message since the client IP would have changed). The server would do something like this:
app.get('/clientupdate',(req,res)=>{
// save off the 'res' object here
res.send('received update');
});
Then later, when the server wants to push data to a client, it would retrieve the saved 'res' object and use it to send the data:
// retrieve the saved 'res' object here
res.send(data);
Is this possible?
Can you instead send both at the same time for example:
var reply = JSON.stringify({
"status": "recieved",
"error": "null",
"response": "data"
}));
res.send(reply);

How to automate API get data request? when using web sockets

As far as I know Web Sockets allows bi-directional communication. and web sockets (for example: Socket.io) connections are always open. so, whenever new data has arrived data should be automatically pushed to the view via socket.
but in below code I am using set_interval to make a http.get call. and set_interval is called once every 1 second.
now, doing these does not give a real-time feel that is, the new data is pulled once every 1 second. which is statically defined.
in-short, I want to automate what set_interval does in below code. I don't want a static fetch interval value. This is because at-times stock price could change within 100ms and at times it would change once in few seconds.
Now, if I set interval to 1 sec, that is make a call every 1 second. the real feel of high fluctuation in market move would not be seen.
I am not sure how usually developers fetch data in IOT applications. for example car is monitored in real-time and let's say speed of the car is fetched in real time and graphed on a web or mobile application.
How do I achieve something similar like that in Stock Ticker? I want to simply plugin the application to an API and when new data arrives instantly push it to all the viewers (subscribers) in real-time.
Code below
////
// CONFIGURATION SETTINGS
////
var FETCH_INTERVAL = 1000;
var PRETTY_PRINT_JSON = true;
////
// START
////
var express = require('express');
var http = require('http');
var https = require('https');
var io = require('socket.io');
var cors = require('cors');
function getQuote(socket, ticker) {
https.get({
port: 443,
method: 'GET',
hostname: 'www.google.com',
path: '/finance/info?client=ig&q=' + ticker,
timeout: 1000
}, function(response) {
response.setEncoding('utf8');
var data = '';
response.on('data', function(chunk) {
data += chunk;
});
response.on('end', function() {
if(data.length > 0) {
var dataObj;
try {
dataObj = JSON.parse(data.substring(3));
} catch(e) {
return false;
}
socket.emit(ticker, dataObj[0].l_cur);
}
});
});
}
I am making a call to method getQuote depending on FETCH_INTERVAL set above
function trackTicker(socket, ticker) {
// run the first time immediately
getQuote(socket, ticker);
// every N seconds
var timer = setInterval(function() {
getQuote(socket, ticker);
}, FETCH_INTERVAL);
socket.on('disconnect', function () {
clearInterval(timer);
});
}
var app = express();
app.use(cors());
var server = http.createServer(app);
var io = io.listen(server);
io.set('origins', '*:*');
app.get('/', function(req, res) {
res.sendfile(__dirname + '/index.html');
});
io.sockets.on('connection', function(socket) {
socket.on('ticker', function(ticker) {
trackTicker(socket, ticker);
});
});
server.listen(process.env.PORT || 4000);
Edits - Update
Okay, so I would need real-time feed. (this bit is sorted)
As far as I know, Real-time feeds are quite expensive and buying 10,000+ end points for each online client is quite expensive.
1) How do I make use of real-time feed to serve 1000s of end users? Can I use web sockets, Redis, publish/subscribe, broadcasting or some technology that copies real-time feed to tonnes of users? I want a efficient solution because I want to keep the expense of real-time data feed as low as possible.
How do I tackle that issue?
2) Yes, I understand polling needs to be done on server side and not on a client-side (to avoid doing polling for each client). but then what tech do I need to use? websockets, redis, pub/sub etc..
I have API URL and a token to access the API.
3) I am not just in need to fetch the data and push it to end users. But I would need to do some computation on the fetched data, will need to pull data from Redis or database as well and do calculations on it then push it to the view.
for example:
1) data I get in real-time market feed {"a":10, "b":20}
2) get data from DB or Redis {"x":2, "y":4}
3) do computation : z = a * x + b * y
4) finally push value of z in the view.
How do I do all these in real-time at the same-time push it to multiple clients?
Can you share a roadmap with me? I got the first piece of the puzzle getting real-time datafeed.
1) How do I make use of real-time feed to serve 1000s of end users? Can I use web sockets, Redis, publish/subscribe, broadcasting or some technology that copies real-time feed to tonnes of users? I want a efficient solution because I want to keep the expense of real-time data feed as low as possible.
How do I tackle that issue?
To "push" data to browser clients, you would want to use a webSocket or socket.io (built on top of webSockets). Then, anytime your server knows there's an update, it can immediately send that update to any currently connected client that is interested in that info. The basic idea is that the client connects to your server as soon as the web page is loaded and keeps that connection open for as long as the web page(s) are open.
2) Yes, I understand polling needs to be done on server side and not on a client-side (to avoid doing polling for each client). but then what tech do I need to use? websockets, redis, pub/sub etc..
It isn't clear to me what exactly you're asking about here. You will get updated prices using whatever the most efficient technology is that is offered by your provider. If all they provide is http calls, then you have to poll regularly using http requests. If they provide a webSocket interface to get updates, then that would be preferable.
There are lots of choices for how to keep track of which clients are interested in which pieces of information and how to distribute the updates. For a single server, you could easily build your own with just a Map of stock prices where the stock symbol is the key and an array of client identifiers is the value in the Map. Then, any time you get an update for a given stock, you just fetch the list of client IDs that are interested in that stock and send the update to them (over their webSocket/socket.io connection).
This is also a natural pub/sub type of application so anyone of the backends that support pub/sub would work just fine too. You could even use an EventEmitter where you .emit(stock, price) and each separate connection adds a listener for the stock symbols they are interested in.
For multiple servers at scale, you'd probably want to use some external process that manages the pub/sub process. Redis is a candidate for that.
3) I am not just in need to fetch the data and push it to end users. But I would need to do some computation on the fetched data, will need to pull data from Redis or database as well and do calculations on it then push it to the view.
I don't really see what question there is here. Pick your favorite database to store the info you need to fetch so you can get it upon demand.
How do I do all these in real-time at the same-time push it to multiple clients? Can you share a roadmap with me? I got the first piece of the puzzle getting real-time datafeed.
Real-time data feed.
Database to store your meta data used for calculations.
Some pub/sub system, either home built or from a pre-built package.
Then, follow this sequence of events.
Client signs in, connects a webSocket or socket.io connection.
Server accepts client connection and assigns a clientID and keeps track of the connection in some sort of Map between clientID and webSocket/socket.io connection. FYI, socket.io does this automatically for you.
Client tells server which items it wants to monitor (probably message sent over webSocket/socket.io connection.
Server registers that interest in pub/sub system (essentially subscribing the client to each item it wants to monitor.
Other clients do the same thing.
Each time client requests data on a specific item, the server makes sure that it is getting updates for that item (however the server gets its updates).
Server gets new info for some item that one or more clients is interested in.
New data is sent to pub/sub system and pub/sub system broadcasts that information to those clients that were interested in info on that particular item. The details of how that works depend upon what pub/sub system you choose and how it notifies subscribers of a change, but eventually a message is sent over webSocket/socket.io for the item that has changed.
When a client disconnects, their pub/sub subscriptions are "unsubscribed".

Websocket transport reliability (Socket.io data loss during reconnection)

Used
NodeJS, Socket.io
Problem
Imagine there are 2 users U1 & U2, connected to an app via Socket.io. The algorithm is the following:
U1 completely loses Internet connection (ex. switches Internet off)
U2 sends a message to U1.
U1 does not receive the message yet, because the Internet is down
Server detects U1 disconnection by heartbeat timeout
U1 reconnects to socket.io
U1 never receives the message from U2 - it is lost on Step 4 I guess.
Possible explanation
I think I understand why it happens:
on Step 4 Server kills socket instance and the queue of messages to U1 as well
Moreover on Step 5 U1 and Server create new connection (it is not reused), so even if message is still queued, the previous connection is lost anyway.
Need help
How can I prevent this kind of data loss? I have to use hearbeats, because I do not people hang in app forever. Also I must still give a possibility to reconnect, because when I deploy a new version of app I want zero downtime.
P.S. The thing I call "message" is not just a text message I can store in database, but valuable system message, which delivery must be guaranteed, or UI screws up.
Thanks!
Addition 1
I do already have a user account system. Moreover, my application is already complex. Adding offline/online statuses won't help, because I already have this kind of stuff. The problem is different.
Check out step 2. On this step we technically cannot say if U1 goes offline, he just loses connection lets say for 2 seconds, probably because of bad internet. So U2 sends him a message, but U1 doesn't receive it because internet is still down for him (step 3). Step 4 is needed to detect offline users, lets say, the timeout is 60 seconds. Eventually in another 10 seconds internet connection for U1 is up and he reconnects to socket.io. But the message from U2 is lost in space because on server U1 was disconnected by timeout.
That is the problem, I wan't 100% delivery.
Solution
Collect an emit (emit name and data) in {} user, identified by random emitID. Send emit
Confirm the emit on client side (send emit back to server with emitID)
If confirmed - delete object from {} identified by emitID
If user reconnected - check {} for this user and loop through it executing Step 1 for each object in {}
When disconnected or/and connected flush {} for user if necessary
// Server
const pendingEmits = {};
socket.on('reconnection', () => resendAllPendingLimits);
socket.on('confirm', (emitID) => { delete(pendingEmits[emitID]); });
// Client
socket.on('something', () => {
socket.emit('confirm', emitID);
});
Solution 2 (kinda)
Added 1 Feb 2020.
While this is not really a solution for Websockets, someone may still find it handy. We migrated from Websockets to SSE + Ajax. SSE allows you to connect from a client to keep a persistent TCP connection and receive messages from a server in realtime. To send messages from a client to a server - simply use Ajax. There are disadvantages like latency and overhead, but SSE guarantees reliability because it is a TCP connection.
Since we use Express we use this library for SSE https://github.com/dpskvn/express-sse, but you can choose the one that fits you.
SSE is not supported in IE and most Edge versions, so you would need a polyfill: https://github.com/Yaffle/EventSource.
Others have hinted at this in other answers and comments, but the root problem is that Socket.IO is just a delivery mechanism, and you cannot depend on it alone for reliable delivery. The only person who knows for sure that a message has been successfully delivered to the client is the client itself. For this kind of system, I would recommend making the following assertions:
Messages aren't sent directly to clients; instead, they get sent to the server and stored in some kind of data store.
Clients are responsible for asking "what did I miss" when they reconnect, and will query the stored messages in the data store to update their state.
If a message is sent to the server while the recipient client is connected, that message will be sent in real time to the client.
Of course, depending on your application's needs, you can tune pieces of this--for example, you can use, say, a Redis list or sorted set for the messages, and clear them out if you know for a fact a client is up to date.
Here are a couple of examples:
Happy path:
U1 and U2 are both connected to the system.
U2 sends a message to the server that U1 should receive.
The server stores the message in some kind of persistent store, marking it for U1 with some kind of timestamp or sequential ID.
The server sends the message to U1 via Socket.IO.
U1's client confirms (perhaps via a Socket.IO callback) that it received the message.
The server deletes the persisted message from the data store.
Offline path:
U1 looses internet connectivity.
U2 sends a message to the server that U1 should receive.
The server stores the message in some kind of persistent store, marking it for U1 with some kind of timestamp or sequential ID.
The server sends the message to U1 via Socket.IO.
U1's client does not confirm receipt, because they are offline.
Perhaps U2 sends U1 a few more messages; they all get stored in the data store in the same fashion.
When U1 reconnects, it asks the server "The last message I saw was X / I have state X, what did I miss."
The server sends U1 all the messages it missed from the data store based on U1's request
U1's client confirms receipt and the server removes those messages from the data store.
If you absolutely want guaranteed delivery, then it's important to design your system in such a way that being connected doesn't actually matter, and that realtime delivery is simply a bonus; this almost always involves a data store of some kind. As user568109 mentioned in a comment, there are messaging systems that abstract away the storage and delivery of said messages, and it may be worth looking into such a prebuilt solution. (You will likely still have to write the Socket.IO integration yourself.)
If you're not interested in storing the messages in the database, you may be able to get away with storing them in a local array; the server tries to send U1 the message, and stores it in a list of "pending messages" until U1's client confirms that it received it. If the client is offline, then when it comes back it can tell the server "Hey I was disconnected, please send me anything I missed" and the server can iterate through those messages.
Luckily, Socket.IO provides a mechanism that allows a client to "respond" to a message that looks like native JS callbacks. Here is some pseudocode:
// server
pendingMessagesForSocket = [];
function sendMessage(message) {
pendingMessagesForSocket.push(message);
socket.emit('message', message, function() {
pendingMessagesForSocket.remove(message);
}
};
socket.on('reconnection', function(lastKnownMessage) {
// you may want to make sure you resend them in order, or one at a time, etc.
for (message in pendingMessagesForSocket since lastKnownMessage) {
socket.emit('message', message, function() {
pendingMessagesForSocket.remove(message);
}
}
});
// client
socket.on('connection', function() {
if (previouslyConnected) {
socket.emit('reconnection', lastKnownMessage);
} else {
// first connection; any further connections means we disconnected
previouslyConnected = true;
}
});
socket.on('message', function(data, callback) {
// Do something with `data`
lastKnownMessage = data;
callback(); // confirm we received the message
});
This is quite similar to the last suggestion, simply without a persistent data store.
You may also be interested in the concept of event sourcing.
Michelle's answer is pretty much on point, but there are a few other important things to consider. The main question to ask yourself is: "Is there a difference between a user and a socket in my app?" Another way to ask that is "Can each logged in user have more than 1 socket connection at one time?"
In the web world it is probably always a possibility that a single user has multiple socket connections, unless you have specifically put something in place that prevents this. The simplest example of this is if a user has two tabs of the same page open. In these cases you don't care about sending a message/event to the human user just once... you need to send it to each socket instance for that user so that each tab can run it's callbacks to update the ui state. Maybe this isn't a concern for certain applications, but my gut says it would be for most. If this is a concern for you, read on....
To solve this (assuming you are using a database as your persistent storage) you would need 3 tables.
users - which is a 1 to 1 with real people
clients - which represents a "tab" that could have a single connection to a socket server. (any 'user' may have multiple)
messages - a message that needs sent to a client (not a message that needs sent to a user or to a socket)
The users table is optional if your app doesn't require it, but the OP said they have one.
The other thing that needs properly defined is "what is a socket connection?", "When is a socket connection created?", "when is a socket connection reused?". Michelle's psudocode makes it seem like a socket connection can be reused. With Socket.IO, they CANNOT be reused. I've seen be the source of a lot of confusion. There are real life scenarios where Michelle's example does make sense. But I have to imagine those scenarios are rare. What really happens is when a socket connection is lost, that connection, ID, etc will never be reused. So any messages marked for that socket specifically will never be delivered to anyone because when the client who had originally connected, reconnects, they get a completely brand new connection and new ID. This means it's up to you to do something to track clients (rather than sockets or users) across multiple socket connections.
So for a web based example here would be the set of steps I'd recommend:
When a user loads a client (typically a single webpage) that has the potential for creating a socket connection, add a row to the clients database which is linked to their user ID.
When the user actually does connect to the socket server, pass the client ID to the server with the connection request.
The server should validate the user is allowed to connect and the client row in the clients table is available for connection and allow/deny accordingly.
Update the client row with the socket ID generated by Socket.IO.
Send any items in the messages table connected to the client ID. There wouldn't be any on initial connection, but if this was from the client trying to reconnect, there may be some.
Any time a message needs to be sent to that socket, add a row in the messages table which is linked to the client ID you generated (not the socket ID).
Attempt to emit the message and listen for the client with the acknowledgement.
When you get the acknowledgement, delete that item from the messages table.
You may wish to create some logic on the client side that discards duplicate messages sent from the server since this is technically a possibility as some have pointed out.
Then when a client disconnects from the socket server (purposefully or via error), DO NOT delete the client row, just clear out the socket ID at most. This is because that same client could try to reconnect.
When a client tries to reconnect, send the same client ID it sent with the original connection attempt. The server will view this just like an initial connection.
When the client is destroyed (user closes the tab or navigates away), this is when you delete the client row and all messages for this client. This step may be a bit tricky.
Because the last step is tricky (at least it used to be, I haven't done anything like that in a long time), and because there are cases like power loss where the client will disconnect without cleaning up the client row and never tries to reconnect with that same client row - you probably want to have something that runs periodically to cleanup any stale client and message rows. Or, you can just permanently store all clients and messages forever and just mark their state appropriately.
So just to be clear, in cases where one user has two tabs open, you will be adding two identical message to the messages table each marked for a different client because your server needs to know if each client received them, not just each user.
As already written in another answer, I also believe you should look at the realtime as a bonus : the system should be able to work also with no realtime.
I’m developing an enterprise chat for a large company (ios, android, web frontend and .net core + postGres backend) and after having developed a way for the websocket to re-establish connection (through a socket uuid) and get undelivered messages (stored in a queue) I understood there was a better solution: resync via rest API.
Basically I ended up by using websocket just for realtime, with an integer tag on each realtime message (user online, typers, chat message and so on) for monitoring lost messages.
When the client gets an id which is not monolithic (+1) then it understands it is out of sync so it drops all the socket messages and asks a resync of all its observers through REST api.
This way we can handle many variations in the state of the application during the offline period without having to parse tons of websocket messages in a row on reconnection and we are sure to be synced (because the last sync date is set just by the REST api, not from the socket).
The only tricky part is monitoring for realtime messages from the moment you call REST api to the moment the server replies because what is read from the db takes time to get back to the client and in the meanwhile variations could happen so they need to be cached and took into account.
We are going into production in a couple of months,
I hope to get back sleeping by then :)
It is seem that you already have user account system. You know which account is online/offline, you you can handle connect/disconnect event:
So the solution is, add online/offline and offline messages on database for each user:
chatApp.onLogin(function (user) {
user.readOfflineMessage(function (msgs) {
user.sendOfflineMessage(msgs, function (err) {
if (!err) user.clearOfflineMessage();
});
})
});
chatApp.onMessage(function (fromUser, toUser, msg) {
if (user.isOnline()) {
toUser.sendMessage(msg, function (err) {
// alert CAN NOT SEND, RETRY?
});
} else {
toUser.addToOfflineQueue(msg);
}
})
Look here: Handle browser reload socket.io.
I think you could use solution which I came up with. If you modify it properly, it should work as you want.
What I think you want is to have a reusable socket for each user, something like:
Client:
socket.on("msg", function(){
socket.send("msg-conf");
});
Server:
// Add this socket property to all users, with your existing user system
user.socket = {
messages:[],
io:null
}
user.send = function(msg){ // Call this method to send a message
if(this.socket.io){ // this.io will be set to null when dissconnected
// Wait For Confirmation that message was sent.
var hasconf = false;
this.socket.io.on("msg-conf", function(data){
// Expect the client to emit "msg-conf"
hasconf = true;
});
// send the message
this.socket.io.send("msg", msg); // if connected, call socket.io's send method
setTimeout(function(){
if(!hasconf){
this.socket = null; // If the client did not respond, mark them as offline.
this.socket.messages.push(msg); // Add it to the queue
}
}, 60 * 1000); // Make sure this is the same as your timeout.
} else {
this.socket.messages.push(msg); // Otherwise, it's offline. Add it to the message queue
}
}
user.flush = function(){ // Call this when user comes back online
for(var msg in this.socket.messages){ // For every message in the queue, send it.
this.send(msg);
}
}
// Make Sure this runs whenever the user gets logged in/comes online
user.onconnect = function(socket){
this.socket.io = socket; // Set the socket.io socket
this.flush(); // Send all messages that are waiting
}
// Make sure this is called when the user disconnects/logs out
user.disconnect = function(){
self.socket.io = null; // Set the socket to null, so any messages are queued not send.
}
Then the socket queue is preserved between disconnects.
Make sure it saves each users socket property to the database and make the methods part of your user prototype. The database does not matter, just save it however you have been saving your users.
This will avoid the problem mentioned in Additon 1 by requiring a confirmation from the client before marking the message as sent. If you really wanted to, you could give each message an id and have the client send the message id to msg-conf, then check it.
In this example, user is the template user that all users are copied from, or like the user prototype.
Note: This has not been tested.
Been looking at this stuff latterly and think different path might be better.
Try looking at Azure Service bus, ques and topic take care of the off line states.
The message wait for user to come back and then they get the message.
Is a cost to run a queue but its like $0.05 per million operations for a basic queue so cost of dev would be more from hours work need to write a queuing system.
https://azure.microsoft.com/en-us/pricing/details/service-bus/
And azure bus has libraries and examples for PHP, C#, Xarmin, Anjular, Java Script etc.
So server send message and does not need to worry about tracking them.
Client can use message to send back also as means can handle load balancing if needed.
Try this emit chat list
io.on('connect', onConnect);
function onConnect(socket){
// sending to the client
socket.emit('hello', 'can you hear me?', 1, 2, 'abc');
// sending to all clients except sender
socket.broadcast.emit('broadcast', 'hello friends!');
// sending to all clients in 'game' room except sender
socket.to('game').emit('nice game', "let's play a game");
// sending to all clients in 'game1' and/or in 'game2' room, except sender
socket.to('game1').to('game2').emit('nice game', "let's play a game (too)");
// sending to all clients in 'game' room, including sender
io.in('game').emit('big-announcement', 'the game will start soon');
// sending to all clients in namespace 'myNamespace', including sender
io.of('myNamespace').emit('bigger-announcement', 'the tournament will start soon');
// sending to individual socketid (private message)
socket.to(<socketid>).emit('hey', 'I just met you');
// sending with acknowledgement
socket.emit('question', 'do you think so?', function (answer) {});
// sending without compression
socket.compress(false).emit('uncompressed', "that's rough");
// sending a message that might be dropped if the client is not ready to receive messages
socket.volatile.emit('maybe', 'do you really need it?');
// sending to all clients on this node (when using multiple nodes)
io.local.emit('hi', 'my lovely babies');
};

Resources