Currently, I'm testing my Node.js, Socket.io server on localhost and on devices connected to my router.
For testing purposes, I would like to simulate a delay in sending messages, so I know what it'll be like for users around the world.
Is there any effective way of doing this?
If it's the messages you send from the server that you want to delay, you can override the .emit() method on each new connection with one that adds a short delay. Here's one way of doing that on the server:
io.on('connection', function(socket) {
console.log("socket connected: ", socket.id);
// override the .emit() method
const emitFn = socket.emit
socket.emit = (...args) => setTimeout(() => {
emitFn.apply(socket, args)
}, 1000)
// rest of your connection handler here
});
Note, there is one caveat with this. If you pass an object or an array as the data for socket.emit(), you will see that this code does not make a copy of that data so the data will not be actually used until the data is sent (1 second from now). So, if the code doing the sending actually modifies that data before it is sent one second from now, that would likely create a problem. This could be fixed by making a copy of the incoming data, but I did not add that complexity here as it would not always be needed since it depends upon how the caller's code works.
An old but still popular question. :)
You can use either "iptables" or "tc" to simulate delays/dropped-packets. See the man page for "iptables" and look for 'statistic'. I suggest you make sure to specify the port or your ssh session will get affected.
Here are some good examples for "tc":
http://www.linuxfoundation.org/collaborate/workgroups/networking/netem
Related
I have run into an unforeseen problem with my socket.io setup.
I use socket.io to live load data from my database (mongoDB, nodejs, react).
To accomplish this, I use mongoDB's changestream to detect changes and then push them to the front-end via socket.io.
Now this works perfectly as long as the user is connected. And right now, when the user reconnects, it just reloads all data. While this is fine for most users, there is a small group with very bad network connection and thus the front-end is reloading data all the time. Which causes the front-end to be unresponsive for some time.
So, I am looking for a way to only send events that occurred during the front-end being offline. While the front-end can do this quite easily: https://socket.io/docs/v4/client-offline-behavior/
It doesn't seem possible to do this at the server side. Since socket.io (server side) immediately forgets sockets that have disconnected and thus cant buffer events.
So, I was wondering if there is a good way do this? Or would this need a full "wrapper" around socket.io that caches disconnected sockets?
Any help or advice would be appreciated!
I find it is a really interesting and painful problem ! ^^'
If you can give more variables, it may help people to give you a better answer
For instance
How many data are stored in database, how much a typical user will receive, and how many events are triggered on a time frame ?
How long should an event take to be visible ? I mean, if users receive an event with a 10s,30s,... delay, is it harmfull for the service they provide.
How your data is structured ? is it a simple json array with the same field, custom field, dynamic json object, etc..
How your react app is structured, do you put heavy logic when your data is update, etc..
I think you should put more controls in your front end code and update only when new datas.
Some paths to explore
1. Put more controls in your front end
As you stated, for the users with bad connection, the react client seems to update his state too quickly, when they reload data after the websocket is connected, again and again. Ui may freeze in this case, yes.
For this, I think of two approaches :
Before updating the state, check if react current state is the same as the data you receive from websocket connection. If the reconnection is quick enough and no new data arrived, it should be the same. So in this case do not update react state.
If too many events are triggered and after each reconnection new data arrived, you can buffer the datas from the websocket and display it only once per time frame. What i mean by time frame, is you can use functions like setInterval or requestAnimationFrame to trigger react update. A pseudo react code to illustrate this.
function App() {
const [events, setEvents] = useState({ datas: [] });
const bufferedEvents = useRef([]);
useEffect(() => {
websocket.on("connected", (newEvents) => {
bufferedEvents.current = bufferedEvents.current.concat(newEvents);
})
websocket.on("data", (newEvent) => {
bufferedEvents.current = bufferedEvents.current.concat(newEvent);
})
// In the setInterval function you take all the events receive at the connection + new events. to update the react state. You clean the bufferedEvents at the same time.
const intervalId=setInterval(() => {
const events = bufferedEvents.current;
bufferedEvents.current = [];
//update if new datas
if (events.length > 0) {
setEvents((prevState) => { return { datas: prevState.datas.concat(events) } });
}
// console.log(events)
}, 1000) // trigger data update every second. You could replace this approach with a requestAnimationFrame. You can adapt the time refresh as you need.
//Do not forget to clear the interval when the component is unmount
return ()=>{
clearInterval(intervalId)
}
}, []);
return (
<div>
<span>Total events : {events.datas.length}</span>
<br />
{
events.datas.map(event => {
return <div>{event.data}</div>
})
}
</div>
)
}
You can look at this article for details on using requestAnimation frame.
I think that modifying the front end is needed in all case, but still alone, not really good on performance.
2. Fetch only new data in your back end
For this approach, it really depends how your data is structured in the database.
If the data have some timestamp in it, I can think of a naive but simple cookie with a timestamp in it.
When user connects the first time, this cookie is null.
When they fetch the data, on the websocket connection, they receive all the datas. When datas arrived, you update the cookie timestamp with the most recent date in the data.
Websocket is disconnected, you open a new websocket with the cookie timestamp on it. With this information you can query all the datas more recent than the timestamp on the cookie.
Like this, you don't have to download the entirity of data, but only fresh ones.
Other approaches may be more helpfull but without more informations on your datas and more precise requirements, it is hard to say.
If you have a lot of data, I will personally check some pagination mechanism and maybe combine some classic http request for fetching the data, and websocket, sse, or long polling for live events.
You can put a comment if needed and I will update my response !
Cheers
When I join the room, and then leave the route and go back, and then use the chat I've built, I get double messages of * amount of messages as many times I left and rejoined.
This problem goes away when I hard refresh.
I've tried everything I could find thus far, and have been unable to get it to work.
I tried on the client side, during beforeRouteLeave, beforeDestroy and window.onbeforeunload
this.$socket.removeListener("insertListener"); --> tried with all
this.$socket = null
this.$socket.connected = false
this.$socket.disconnected = true
this.$socket.removeAllListeners()
this.$socket.disconnect()
During the same events, I also sent a this.$socket.emit("leaveChat", roomId) and then on the server side tried the following inside the io.on("connection") receiver socket.on("leaveChat", function(roomId) {}):
socket.leave(roomId) --> this is what should according to docs work;
socket.disconnect()
socket.off() -- seems to be deprecated
socket.removeAllListeners(roomId)
There were a bunch of other things I tried that I can't remember but will update the post if I do.
Either it somehow disconnects and upon rejoining, previous listeners or something is still remaining, meaning all the messages are received * times rejoin. OR, if I disconnect, I don't seem to be able to reconnect.
On joining, I emit to server the room id and use socket.join(roomId).
All I want to do, is without refresh, when I leave the page, before that happens, the user can leave the room and when they go back, they get to rejoin, with no duplicate messages occurring.
I am currently trying to chew through the source code now.
Full disclosure here, I didn't read the full response posed by roberfoenix, but this is a common issue with socket.io and it comes down to calling the 'on' event multiple times.
When you create an .on event for your socket its a bind, and you can bind multiple times to the same event.
My assumption is, when a users hits a page you run something like
socket.on("joinRoom", data)
This in turn will say join the room, pull your messages from Mongo(or something else) and then emit to the room (side note, using .once on can help so you don't emit to every users when a user joings a room)
Now you leave the room, call socket.emit('leaveRoom',room), cool you left the room, then you go back into the room, guess what you now just binded to the same on event again, so when you emit, it emits two times to that user etc etc.
The way we addressed this is to place all our on-events into a function and call the function once. So, a user joins a page this will run the function like socketInit();
The socketInit function will have something like this
function socketInit(){
if (init === false){
//Cool it has not run, we will bind our on events
socket.on("event")
socket.on("otherEvent")
init = true;
}
}
Basically the init is a global variable, if is false, bind your events, otherwise don't rebind.
This can be improved to use a promis or could be done on connect but if a users reconnects it may run again.
If you're using Vue-Socket and feel like going slightly mad having tried everything, this may be your solution.
Turns out challenging core assumptions and investigating from the ground up pays off. It is possible that you forgot yourself so deeply in Socket.io, that you forgot you were using Vue-Socket.
The solution in my case was using Vue-Socket's built in unsubscribe function.
With Vue-Socket, one of the ways you can initially subscribe to events is as follows:
this.sockets.subscribe('EVENT_NAME', (data) => {
this.msg = data.message;
});
Because you're using Vue Socket, not the regular one, you also need to use Vue Socket's way for unsubscribing right before you leave the room (unless you were looking for a very custom solution). This is why I suspect many of the other things I tried didn't work and did next to nothing!
The way you do that is as follows:
this.sockets.unsubscribe('EVENT_NAME');
Do that for any events causing you trouble in the form of duplicates. The reason you'd be getting duplicates in the first place, especially upon rejoining post leaving a room, is because the previous event listeners were still running, and now the singular user would be playing the role of as if two or more listeners.
An alternative possibility is that you're emitting the message to everyone, including the original sender, when you should most likely be emitting it to everyone else except the sender (check here for socket.io emit cheatsheet).
If the above doesn't solve it for you, then make sure you're actually leaving the room, and doing so server-side. You can accomplish that through emitting a signal to the server right before leaving the route (in case you're using a reactive single page application), receiving it server side, and calling 'socket.leave(yourRoomName)' inside your io.on("connection", function(socket) {}) instance.
I am unable to get two users chatting to each other despite reducing the complexity and the potential code that could have caused the issue.
I am able to emit to all connected sockets so I have established it's not an issue in context of emit/on structure but rather; coming from the way i'm handling the private socket ids.
I have tried various versions of trying to send the private message to the correct socket id; I have tried older ways such as socket.to and the current way from the docs which is io.to(sockid).emit('event' message); all these variations have been unable to help me. I have consoled out the socket id I have on my Angular client side by printing console.log('THIS IS MY SOCKET '+this.socket.id) and comparing it to the value I have in Redis using redis-cli and they both match perfectly every time which doesn't give me too much to go on.
problem arises here:
if (res === 1) {
_active_users.get_client_key(recipient)
.then(socket_id => {
console.log('======='+io.sockets.name)
console.log('I am sending the message to: '+ recipient + 'and my socket id is'+ socket_id)
// socket.to(socket_id)socket.emit('incoming', "this is top secret"
io.of('/chat').to(socket_id).emit('incoming', "this is top secret")
})
.catch(error => {
console.log("COULD NOT RETRIEVE KEY: " + error)
})
Here is the link to the pastebin with more context:
https://pastebin.com/fYPJSnWW
The classes I import are essentially just setters and getters for handling the socket id you can think of them as just a worker class that handles Redis actions.
Expected: To allow two clients to communicate based on just their socket ids.
Actual:
am able to emit to all connected sockets and receive the expected results but the problem arises when trying to send to a specific socket id from a unknown reason.
Issue was coming from my front end.. I hope nobody has a headache like this! but here is what happened; when you're digging your own hole you often don't realise how deep you got yourself if you don't take the time to look around. I had two instances of the sockets. I instantiated both and used the one to connect and the other to send the message; which of course you cannnot do if you want things to work properly. So what I did was created only one instance of the socket in and and passed that ref of the socket around where I needed it which is ( sendMessage(username, socket) getMessage(socket)
ngOnInit(
this.socket = io.connect('localhost:3600',{
reconnection: true,
reconnectionDelay: 1000,
reconnectionDelayMax : 5000,
reconnectionAttempts: Infinity});
I'm trying to develop an API for multiplayer online using socket programming in node js
I have some basic questions:
1. How to know which connection is related to a user?
2. How to create a socket object related to another person?
3. When it's opponent turn, how to make an event?
4. There is a limited time for move, how to handle the time to create an event and change turn?
As it is obvious I don't know how to handle users and for example list online users
If you can suggest some articles or answering these questions would be greate
Thanks
Keep some sort of data structure in memory where you are saving your sockets to. You may want to wrap the node.js socket in your own object which contains an id property. Then you can save these objects into a data structure saved in memory.
class User {
constructor(socket) {
this.socket = socket;
this.id = //some random id or even counter?
}
}
Then save this object in memory when you get a new socket.
const sockets = {}
server = net.createServer((socket) => {
const user = new User(socket);
sockets[user.id] = user
})
I am unsure what you mean by that, but maybe the above point helps out
This depends on when you define a new turn starts. Does the new turn start by something that is triggered by another user? If so use your solution to point 2 to relay that message to the related user and write something back to that socket.
Use a timeout. Maybe give your User class an additional property timeout whenver you want to start a new timeout do timeout = setTimeout(timeouthandler,howlong) If the timeouthandler is triggered the user is out of time, so write to the socket. Don't forget to cancel your timeouts if you need to.
Also, as a side note, if you are doing this with pure node.js tcp sockets you need to come up with some ad-hoc protocol. Here is why:
socket.on("data", (data) => {
//this could be triggered multiple times for a single socket.write() due to the streaming nature of tcp
})
You could do something like
class User {
constructor(socket) {
this.socket = socket;
this.id = //some random id or even counter?
socket.on("data", (data) => {
//on each message you get, find out the type of message
//which could be anything you define. Is it a login?
// End of turn?
// logout?
})
}
}
EDIT: This is not something that scales well. This is just to give you an idea on what can be done. Imagine for some reason you decide to have one node.js server instance running for hundreds of users. All those users socket instances would be stored in the servers memory
I've a node.js (tcp) server, i.e.
net.createServer
And I've a node.js client. I've created a module.export with a method
this.connect = function() {
var client = new net.Socket();
client.setTimeout(5000);
client.connect(settings.collectorport, settings.collectorhost, function() {
console.log('connected');
});
....
And another method, like
this.write = function(data, client) {
client.write(some-string);
}
From another node.js file, I invoke such methods from inside a loop like:
while (something) {
agent.write(data,client);
}
What happens is that sometimes the server receives, or the client send, two "some-string" all togheter.
I mean, if in the server I log what the server is receiving, I see, sometimes only one "message", sometimes two or three "messages" merged.
What can it be?
Sorry for my lexicon...
Assuming the number of messages you are sending matches the number of messages you are receiving/expecting on the other end, then I think what you're describing is basically how TCP works.
TCP is a stream and so you can't/shouldn't make any assumptions about "message boundaries." You could receive one byte or 10 kilobytes at a time, it just depends on the networking stack and some other factors, so you should be prepared to handle any amount of data that comes in (e.g. create a protocol, perhaps using a delimiter character, to sort out individual "messages").