I am working on an client/server node+react application that displays logs. I want the user to have a constant stream of data flowing in, or at least appear to be. Is the most efficient way to use a Websocket in node, and just connect the client to it?
If you're only streaming data from the server to the client, and that data is text, there is no need for Web Sockets.
Server-Sent Events (SSE) and the EventSource API are a simpler choice. It's specifically designed to update the client as things happen, and sounds like a good fit for your use case. They remain connected, will auto-reconnect if the connection is lost, and can support resuming from where they left off.
Web Sockets are more appropriate for when you want bi-directional data streaming.
Related
I wonder how do I use socket.io properly with my express app.
I have a REST API written in express/node.js and I want to use socket.io to add real-time feature for my app. Consider that I want to do something I can do just by sending a request to my REST API. What should I do with socket.io? Should I send request to the REST API and send socket.io client the result of the process or handle the whole process within socket.io emitter and then send the result to socket.io client?
Thanks in advance.
Question is not that clear but from what I'm getting from it, is that you want to know what you would use it for that you cant already do with your current API?
The short answer is, well nothing really.. Websockets are just the natural progression of API's and the need for a more 'real-time' interface between systems.
Old methods (and still used and relevant for the right use case) is long polling where you keep checking back to the server for updated items and if so grab them.. This works but it can be expensive in terms of establishing a connection, performing a lookup, then closing a connection.
websockets keep that connection open, allowing both the client and server to communicate real time. So for example, lets say you make an update to your backend data and want users to get that update, using long polling you would rely on each client to ping back to the server, check if there is an update and if so grab it. This can cause lags between updates, some users have updated data while other do not etc.
Now, take the same scenario with websockets, you make an update to the backend data, hit submit, this then emits to your socket server. Socket server takes the call, performs the task ( grabs updated data ) and emits it to the users, each connected user instantly gets that update.
Socket servers are typically used for things like real time chats or polling where packets are smaller but they are also used for web games etc. Depending on the size of your payloads will determine how best to send data back and forth because the larger the payload the more resources / bandwidth it will take on the socket server so its something to consider.
I want to make an app which lets users comment and send messages. However, the notifications for these events will have to come instantly, just like any other social-media or chat application. This is what I'm thinking of:
Web-frontend: Angular, mobile: Ioinc with Angular
Backend: Node, Mongo
Now, this is how I was thinking I'd implement real-time notification.
There's a constant socket connection between the frontend (web & mobile-app) and the backend.
Whenever a message arrives, targeted to a specific user, I'll use some kind of a Mongo-hook to send the notification to the frontend via the socket connection.
Now, the confusion with this approach is:
Would millions of socket connections work at scale, at all? If not, what is the way to implement this pub-sub kind of system? I need to do it from scratch, not using Firebase.
What if a user is offline when he receives the message in the backend? If the socket is not on, how would he get the message? Is there a way to do it using Kafka? Please explain if you have some ideas on this.
Is this the correct approach? If not, can you suggest what would be appropriate?
Would millions of socket connections work at scale, at all? If not, what is the way to implement this pub-sub kind of system? I need to do it from scratch, not using Firebase.
Yes, it can work at scale just you have to made an architecture like that. You might find this useful
Scalable architecture for socket.io
https://socket.io/docs/v3/using-multiple-nodes/
What if a user is offline when he receives the message in the backend? If the socket is not on, how would he get the message?
If he the socket is not on or user is offline, then client Socket will be disconnected. At this point, notification will not be received and whenever the user comes online you'll have make an API call to get the notifications and connect again to the socket for further operations.
Is there a way to do it using Kafka?
Yes, you can also do it with Kafka. You'll need Consumer API(Subscriber) and Producer API(Publisher)
https://kafka.apache.org/documentation/#api
https://www.npmjs.com/package/kafka-node
Sending Apache Kafka data on web page
What do you use Apache Kafka for?
Real time notification with Kafka and NodeJS
I am in the process of developing a chat application using Javascript. When sending messages from one client to another client, do I have to send it through a server or can I send it directly from a peer-to-peer approach, using something like websockets ?
Welcome to the stage of life where you see the importance of design patterns.
You can start solutionizing with mediator pattern and proxy pattern with web sockets.
Wheater you need a server or not is up to your design.
Technology-wise there are multiple APIs that HTML5 offers you can go through them and make something on your own.
There is a bunch of APIs available with HTML5 and JS.
Start digging on WebSockets, Server-Sent Events, Web Workers.
The server will give you the flexibility of record-keeping while acting as a mediator. Alternatively, you can come up with a pure p2p design with a scheme where every node or user notify other users with their details(IP) for establishing communication. Remember for web socket to work the client need to know what address to connect to. Maybe it can have fixed master nodes. Then you can use observables for polling and other features. Take a look at the BitTorrent protocol for design inspiration.
Get creative and start designing.
There are many ways to do it. I recommend the scheme:
Peer <---> custom websocket server <---> Peer;
I recommend NodeJS with SocketIO.
Is it possible to implement a two-way communication between client and server with Apache Thrift? Thus not only to be able to make RPC from client to server, but also the other way round? In my project I have the requirement that the server must also push some data to the client without being asked by the client before to do this.
There are two ways how to achieve this with Thrift.
If both ends are more or less peers and you connect them through sockets or pipes, you simply set up a server and a client on both ends and you're pretty much done. This does not work in all cases, however, especially with HTTP.
If you connect server and client through HTTP or a similar channel, there is a technique called "long polling". It basically requires the client to call the server as usual, but the call will only return when the server wants to send some data back to the client. After receiving the data, the client starts another call if he's still interested in more data.
As Denis pointed out, depending on your exact use case, you might want to consider using a MQ system. Note that it is still possible to use Thrift to de/serialize the messages into and from the queues. The contrib folder has some examples that show how to use Thrift with ZMQ, Rebus and some others.
You are better to use queues then, e.g. ZeroMQ.
Background: I am building a web app using NodeJS + Express. Most of the communication between client and server is REST (GET and POST) calls. I would typically use AJAX XMLHttpRequest like mentioned in https://developers.google.com/appengine/articles/rpc. And I don't seem to understand how to make my RESTful service being used for Socket.io as well.
My questions are
What scenarios should I use Socket.io over AJAX RPC?
Is there a straight forward way to make them work together. At least for Expressjs style REST.
Do I have real benefits of using socket.io(if websockets are used -- TCP layer) on non real time web applications. Like a tinyurl site (where users post queries and server responds and forgets).
Also I was thinking a tricky but nonsense idea. What if I use RESTful for requests from clients and close connection from server side and do socket.emit().
Thanks in advance.
Your primary problem is that WebSockets are not request/response oriented like HTTP is. You mention REST and HTTP interchangeably, keep in mind that REST is a methodology behind designing and modeling your HTTP routes.
Your questions,
1. Socket.io would be a good scenario when you don't require a request/response format. For instance if you were building a multiplayer game in which whoever could click on more buttons won, you would send the server each click from each user, not needing a response back from the server that it registered each click. As long as the WebSocket connection is open, you can assume the message is making it to the server. Another use case is when you need a server to contact a client sporadically. An analytics page would be a good use case for WebSockets as there is no uniform pattern as to when data needs to be at the client, it could happen at anytime.
The WebSocket connection is an HTTP GET request with a special header requesting the server to upgrade it to a WebSocket connection. Distinguishing different events and message on the WebSocket connection is up to your application logic and likely won't match REST style URIs and methods (otherwise you are replication HTTP request/reply in a sense).
No.
Not sure what you mean on the last bit.
I'll just explain more about when you want to use Socket.IO and leave the in-depth explanation to Tj there.
Generally you will choose Socket.IO when performance and/or latency is a major concern and you have a site that involves users polling for data often. AJAX or long-polling is by far easier to implement, however, it can have serious performance problems in high load situations. By high-load, I mean like Facebook. Imagine millions of people loading their feed, and every minute each user is asking the server for new data. That could require some serious hardware and software to make that work well. With Socket.IO, each user could instead connect and just indefinitely wait for new data from the server as it arrives, resulting in far less overall server traffic.
Also, if you have a real-time application, Socket.IO would allow for a much better user experience while maintaining a reasonable server load. A common example is a chat room. You really don't want to have to constantly poll the server for new messages. It would be much better for the server to broadcast new messages as they are received. Although you can do it with long-polling, it can be pretty expensive in terms of server resources.