I wonder how do I use socket.io properly with my express app.
I have a REST API written in express/node.js and I want to use socket.io to add real-time feature for my app. Consider that I want to do something I can do just by sending a request to my REST API. What should I do with socket.io? Should I send request to the REST API and send socket.io client the result of the process or handle the whole process within socket.io emitter and then send the result to socket.io client?
Thanks in advance.
Question is not that clear but from what I'm getting from it, is that you want to know what you would use it for that you cant already do with your current API?
The short answer is, well nothing really.. Websockets are just the natural progression of API's and the need for a more 'real-time' interface between systems.
Old methods (and still used and relevant for the right use case) is long polling where you keep checking back to the server for updated items and if so grab them.. This works but it can be expensive in terms of establishing a connection, performing a lookup, then closing a connection.
websockets keep that connection open, allowing both the client and server to communicate real time. So for example, lets say you make an update to your backend data and want users to get that update, using long polling you would rely on each client to ping back to the server, check if there is an update and if so grab it. This can cause lags between updates, some users have updated data while other do not etc.
Now, take the same scenario with websockets, you make an update to the backend data, hit submit, this then emits to your socket server. Socket server takes the call, performs the task ( grabs updated data ) and emits it to the users, each connected user instantly gets that update.
Socket servers are typically used for things like real time chats or polling where packets are smaller but they are also used for web games etc. Depending on the size of your payloads will determine how best to send data back and forth because the larger the payload the more resources / bandwidth it will take on the socket server so its something to consider.
EDIT:
I started nodejs few days ago and i want to understand one thing.
Imagine that i have a nodejs web communicate with API, when i send request (offer for a specific user) i would like to know if user has accepted or declined the offer without refreshing the site. I know there is a way with AJAX, but is there any better solution how can i get state of the offer if it is accepted/declined (if something change)
Every advice is appreciated!
If you want to know in the client when the state of something on the server changes (at some indefinite future time), then here are three options:
You can regularly "poll" the server every so often with an ajax call asking for any updates on the offer. The server can then return the current state of the offer and the client can update the status in the current web page.
You can create a webSocket or socket.io (socket.io is an API on top of a webSocket) connection from the client to the server. This is a long lasting connection which allows the server to send data to the client at any time. So, anytime the server sees a change in the state of that offer, it can send an update to the client and the client can then modify the current page to show that change.
You can use the newer server-sent events which is an extension to http which allows a server to send data to a client to accomplish something similar to the previous option.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 5 years ago.
Improve this question
I am making a chat app and would like to add the functionality of showing an online/offline symbol next to users. How can I do this reliably with minimum number of srever requests and database writes?
One way I found was to update a lastSeenAt field in the user document every time that user requests a page and use this to indicate whether the user is online/offline. Another way is to ping the server from the client side at fixed intervals of time and then update the lastSeenAt field.
Both theses ways would require a lot of database writes and/or server requests. Is there a way to do this more efficiently?
You have to push data from your server to the clients upon change, with something like web-sockets or server sent events. socket.io is a popular tool for implementing such functionally.
With socket.io or specifically websockets you could track the if a client is connected or not and have a flag in database that tracks whenever a client connects or
disconnects(keep in mind, the client might be having multiple connections(multiple devices, or even browser tabs), so if one connection disconnect he still might be online.
Assuming you will have multiple state-less webservers(common practice), then once a client connects or disconnects you want to notify other interested clients, then you should a use a pubsub pattern to notify other servers which will notify their connected clients respectively. There's a lot of implementations such as Redis, zeromq, AWS SNS, GCP Cloud Pubsub. You could even use MongoDB with tailable cursors as your pubsub.
However this might mean a lot of constant connections to your server, so this might hurt your scaliblity. If it proves to be to expensive for you then your lastSeenAt approach with some sane polling. You can find out which works better with your setup by just running some experiments.
If database writes and requests worries, you could always throw more servers at it. You could have a microservice specifically for this so your main server wouldn't be affected much and also you could have a database specifically for this. If performance worries you, you can use a in-memory database.
Also you can also have some caching in your servers with simple time based invalidation or you can sync the online/offline data between your servers with a pubsub pattern.
I would suggest try to make your setup as simple as possible, and you can run experiments to see how it handles your expected traffic.
Look into Express.io where you can combine HTTP calls with sockets. (the best method for real-time communication is Sockets).
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 7 years ago.
Improve this question
I have a game server where clients can connect and communicate with via TCP. Any device can connect the server if it knows the IP and port.
I am wondering if I need to add some security to the server. For example,
(1) Add some encryption for the messages sent/received. (To prevent the protocol content is revealed)
(2) Add some key to the message so if the server cannot recognize the key after decryption, the message will be dropped. (To prevent unknown connections/messages flooding in)
Do you think these things are necessary and is there any other thing I should add for such a game server.
I would have rather posted this to the gamedev question but the mods there are apparently faster than here. Before you quote me, I'd like to point out that the following isn't based on 100% book-knowledge, nor do I have a degree in any of these topics. Please improve this answer if you know better, rather than comment and/or compete.
This is a pretty comprehensive list of client/server/security issues that I've gathered from research and/or experience:
Data
The "back-end" server contains everyone's username, password, credit card details, etc., and should be a fortress. This server is for authentication only and should be on a private subnet; it will communicate only with the login server, only when a well-formed login request is received, and will only reply with "allow" or "deny". If you take people's personal information, you are obligated to protect it, and it would be wise to off-load the liability of everything security-related to a professional or hosting company. There is no non-critical attack to this server; if it is breached, you are finished. Many/most/all companies now draw their pretty login screen on top of another companies' back-end credit card/billing system.
Login
Connections to the login server should be secure. The login server is just a message pump between the public login mechanism, the private data store, and the client/server connection state. For security purposes, any HTTP access to the login system should be hosted on a separate HTTP server; the WWW server crashing should not shut down your online game (my opinion).
World/UDP
Upon successful login and authentication, the server informs the client to begin listening for "bulk data" or to initiate an in-bound connection on a specific UDP port (could be random and per-connection-attempt). Either way, the server should remain silent and wait for the client to IDENT with some type of handshake to verify that the "alleged client" is actually your code. It is easier to guess when the server asks for input sequentially; instead rely on the client knowing the proper handshake when connecting to the world and drop those that don't. The correct handshake to use can be a function of the CPU clock-ticks or whatever. The TCP will be minimally used and/or disconnected from that point on. The initial bulk data is a good place to advertise the current server-side software revision so clients that are out-of-date can update. A common pool of UDP ports can be handed out among multiple servers and the clients can be load-balanced into the correct port/server. Within the game, "zone transfers" can mean a literal disconnect from one server/port and reconnection to a different server/port. In MMO's, this usually appears as a <2 second loading screen; enough time to disconnect, reconnect, start getting data, and synchronize to the new server clock, not to mention the actual content loading.
"World server" describes a single, multiple-client, state-pumping thread running on a single core of a single processor of a single blade. One, physical, server-of-worlds can have many worlds running on it at once. Worlds can be dynamically split/merged (in a quad-tree fashion), dividing the clients between them, again, for load-balancing; synchronization between the servers occurs at LAN speeds or better. The world server will probably only serve UDP connections and should have nothing to do except process state-changes to/from the UDP connections. UDP is "blind, deaf, and dumb", so-to-speak. Messages are sent with no flow control, no error checking, etc; they are basically assumed to be received as soon as they are sent and may actually arrive late, in the wrong order, or just never arrive. Using UDP, neither the server nor the client are ever stalled, hand-shaking, error-correcting, or waiting for data. Messages need time-stamps because they may arrive late and/or out-of-order. If a UDP channel gets clogged, switch valid clients dynamically to another (potentially random) port. The world server only initiates UDP connections with successfully authenticated clients and ignores all other traffic (world servers hosted separately from HTTP and everything else).
Overly simplified and, using only the position data as an example, each client tells the server "Time:Client###:(X, Y)" over and over. If the server doesn't hear, oh well. The server says "Time:listOfClients(X, Y)" over and over, to everyone at once. If one or more of the clients doesn't hear, oh well.
This implies using prediction/extrapolation on the client; the clients will need to "guess" what should be happening and then correct themselves to agree with the server when they start getting data again. Any time you get a packet with a "future" time, even if the packet doesn't make sense or isn't useful, you can at least advance the client clock to that point and discard any now-late packets, helping a lagging client to catch up.
Un-verified supposition:
Besides the existing security concerns, I don't see a reason why two or more clients could not maintain independent, but server-managed, UDP channels between each other. By notifying other clients within close game-proximity in addition to the server, the clients, themselves, can help to load-balance. The server should always verify that what the clients say happened could/should/would happen, and has the ability to undo all of it and reset both clients to it's own known-good state. The information that the clients are able to share, internally, should be extremely restricted; basically just the most-time-critical positional and/or state-data. Client's should probably not be allowed to request specific information and, again, rely only on "dumb" broadcasts. This begins to approach distributed/cloud computing, where the clients are actually doing a lot of the server work, while the server just watches and "referees," calling foul, when appropriate.
Client1 - "I fought Client2 and won"
Client2 - "I fought Client1 and won"
Server - "I watched and Client2 cheated. Client1 wins. (Client2 is forced to agree)"
The server doesn't necessarily even need to watch; if Client2 damages Client1 in an unusual/impossible way, Client1 can request arbitration from the server.
Side-effects
If the player moves around, but the data isn't getting to the server, the player experiences "rubber-banding", where the player appears to be moving on the client but, server-side, they are not. When the client gets the next server state, the client snaps the player back to where they were when the server stopped getting updates, creating the rubber-band effect.
This often manifests another way, too. If the server sees a player moving, then fails to receive the "stopped moving" message, the server will predict their continued path for all of the other clients. In MMO-RPG's, for example, you can see "lagging" players running directly into/at walls.
Holes
The last thing I can think of is just basic code security. This is especially important if your game is moddable. Mods are, by definition, a way for users to insert their own code into yours. If you are careless about the amount of "API" access you give away, inevitably, someone WILL feel the need to be malicious. Pay particular attention to string termination/handling if the language you are using requires it. Do not build your game from plain-text ASCII content files. If your game has even one "text box," someone WILL be trying to feed HTML/LUA/etc. code into it.
Lastly, paths should use appropriate system variables whenever possible to avoid platform shenanigans and/or access violations (x86/x64, no savegames in ProgramFiles, etc.)
So on a site like, say, stack overflow parts of the page update when things happen like your reputation increase. How do they do that lol? Does a script check from time to time or is it a push notification somehow?
About 2 years ago stackexchange started using web sockets as stated here:
https://meta.stackexchange.com/questions/125677/new-feature-real-time-updates-to-questions-answers-and-inbox
If you take a look at the stackoverflow site source you will see that a JavaScript function subscribes to a web socket server.
There are many different approaches to that technology now. Microsoft for example introduced SignalR (http://signalr.net/) which degrades gracefully to older browser too by switching to other technologies where sockets don't work like long polling (asking every X seconds if changes are available).
You as a Python guy would probably start looking at something like: https://pypi.python.org/pypi/websockets/1.0
Have fun with web sockets!
If I didn't want to use web sockets, I would do it like this:
Have the server maintain a queue of notifications for a session or user or whatever context you want.
Have a URL for fetching such notifications.
When a client tries to GET that URL, and there are notifications available in the queue, return them immediately.
Otherwise, have the HTTP connection block until there are notifications queued.
On the client, side, then; simply try to GET the notification URL over and over again. Normally, the connection will sit blocking for data to read, but I don't see that this should be a problem.
I would think this should be easier to implement on the server side than web sockets are, since the HTTP server doesn't have to support any special HTTP extensions. On the other hand, depending on the HTTP server you're using, each such open connection may be using a thread or other system resource that you want to use sparingly.