I have a structure where an application sends a POST request, the API handles the parameters and enters them into the database then another application that is supposed to run a function when the POST request was handled. The only issue I'm having is how can I make the 2nd application know when the request was handled other than running a GET request on a time interval?
Another explanation:
Client A: POSTs data
API: Handles data
Client B: GETs and displays data to user
How can I tell Client B when to do the GET request?
This is all in node.js and using express
What you need is to push data to clients when something happens server-side. This can be achieved by either server side events (which do exactly this), or websockets which create a bidirectional communication channel between server and client. Which one to choose? Check out this stack overflow post.
Related
I'm creating a chat application, and one detail is that "acknowledgements" are crucial. I'll get to what that means. I'm trying to figure out what the best exchange protocol would be.
Scenario:
Alice sends Bob a message. Bob is offline, so the message is stored on the server. Bob connects to the server through a WebSocket connection. The server sends him messages that have been sent to him while he was away. This is where the problem arises. The WS API that's available for my app's ecosystem (Node.js, Nest.js specifically), has no pattern where it can wait for this message to be sent. The mechanism there seems to just be fire & forget. What if the payload is quite large and the connection drops while the message is being sent?
Now, I know socket.io has support for acknowledgements. But from what I've read, socket.io has some overhead and therefore less performance than optimal. Now whether that performance is something that I arguably need is another question, but I'm just trying to figure out how I can guarantee that the message has arrived on the other end. This means client-server and server-client directions. How can I await it? I know that one approach is to attach a unique ID to the socket event, and have the other side send you a confirmation that it received it. This is how socket.io does it if I'm not mistaken.
But my question there is how can I guarantee that the acknowledgement message was successfully sent? So then I'd need an "ack" for my "ack" and so on, so I'll always need one more acknowledgement so I don't know how that works.
What I though of as options is to use two REST endpoints to send and receive (or download) messages. You send when you send, but you receive when you receive a ping that there's messages for you to download. Now this could be done through a WebSocket connection where the server notifies the client about a new message and then the client can call this receive endpoint. This ping can also be done through a more managed solution like FCM. The pros with that approach are twofold:
First, I have the REST interface to use, which is a lot more practical
I have the Request-Response pattern to use, so I have a theoretical guarantee that things are arriving if I get a response
Now the problem with this approach is that there's a lot of overhead from opening a new HTTP connection every time I want to send or receive messages, if I'm not mistaken:
I have to wait for the initial request time to get to the server before I actually have to wait for the server to respond with messages. With the pure WebSockets case, I would theoretically then just wait for the response equivalent part there (?)
This wastes bandwidth as well.
So one more question, where can I find out which clients will actually re-use an existing HTTP connection like a WebSocket connection, if available and not create a new one? Do all clients do that? Is it only the browser? What about apps? Is it on the OS level?
So the final question is how do I solve this problem of "acknowledgements" and not waste time and bandwidth? Are any of my conclusion/questions wrong or uninformed, am I missing something?
Notes:
server is Node.js and client is Flutter
i know about the WAMP subprotocol, but for my ecosystem it doesn't have very reliable implementations
I'm not sure what your exact requirements or performance need,
but I did a project that also need reliable communication between client and server using websocket, the simplest I could think of was build request-response mechanism on top of websocket, and then build your application data on top of that.
here's high level overview how I implemented it:
implement request-response message using transaction to identify which response belongs to which request.
clients will have map storing transaction, when you send the message request wait for server to send a message response with the same transaction or wait
until timeout.
client wants to send message to server and construct the request as follow
{
"event": "sendMessage",
"type": "request",
"transaction": "<uuid/unique-value>",
"data": "<your-application-data>"
}
server parse the message and check that its a request with an event name sendMessage then call related function
server sends back response message to client
{
"event": "sendMessage",
"type": "response",
"transaction": "<uuid/unique-value>", // same unique value as in request
"data": "<your-application-data-result>"
}
because client has mapping which transaction belongs to which request, it is possible to match which request this response belongs to, if matched then complete the transaction
I'm working on a React/Node program. Somewhere in the program, I want to send a request to back-end using Axios, then while the back-end function is processing the request, I want to get step by step responses to update the front-end by showing some notifications.
I could do this by sending multiple requests and waiting for each response. But the problem is that the first process in each step is identically the same in all steps and it will create some performance issues.
My question is:
Is there any way to send a single request to API, then on the back-end side, return the response in multiple steps while it's processing? Meanwhile on the front-end, get the updates from back-end and update the notifications on it?
Thank you very much
Sorry bro, I'm afraid that you can't do this with using HTTP alone since the connection is ended with a single response for a single request. You need to do this with multiple HTTP call with Axios.
Otherwise, you could use WebSocket.
Their are cool Module socket.io with planty examples and documentations.
check this out,
https://www.npmjs.com/package/socket.io
I've got an chatbot app where I want to send one message e.g. res.json("Hello") from express, then another message later e.g. res.json("How are you doing"), but want to process some code between the two.
My code seems to have some problems with this, because when I delete the first res.json() then the second one works fine and doesn't cause any problems.
Looking in my heroku logs, I get lots of gobbledy gook response from the server, with an IncomingMessage = {}, containing readableState and Server objects when I include both of these res.json() functions.
Any help would be much appreciated.
HTTP is request/response. Client sends a request, server sends ONE response. Your first res.json() is your ONE response. You can't send another response to that same request. If it's just a matter of collecting all the data before sending the one response, you can rethink your code to collect all the data before sending the one response.
But, what you appear to be looking for is "server push" where the server can send data to the client continually whenever it wants to. The usual solution for that is a webSocket connection (or socket.io which is built on top of webSocket and adds more features).
In the webSocket/socket.io architecture, the client makes a connection the server and the connection is kept open indefinitely. Then either side of the connection can send messages to the other end. This is most useful when the server wants to "push" data to the client at any time. In this case, the client establishes the connection, then the server can send data to the client over that connection at any time. The client registers a listener for incoming messages and will be notified anytime the server sends it some data.
Both webSocket and socket.io are fully supported in modern browsers and in node.js. I would personally recommend using socket.io because some of the features it adds (a messaging layer, auto-reconnect, etc...) are very useful.
To use a continuously connected socket like this, you will have to make sure your hosting infrastructure is properly configured to allow it.
res.json() always sends the response to the client immediately (calling it again will cause an error). If you need to gradually build up a response then you can progressively decorate a plain old javascript object; for example, appending items to an array. When you are done call res.json() with the constructed response.
But you should post your code so we can see what's happening.
Let me clarify the title of the question.
On the client side I have two different html files, client1.html and client2.html. They send data via websockets to the same server.js file, in node.js.
On the server side, if the data came from client1.html I want to perform a query and send the outcome to the client1.html. On the same server, if the data came from client2.html I want to perform another query and send to the client2.html the message "Data Saved".
I guess, I have to create two different functios in the server.js. OK.
But, my problem is, on the server side, how to tell, which data came from which client?
Also, how the server can send back the right message to the right client?
Thanks in advance
You have to register your clients. For example if a user A is on page client1.html then you send a message (via websockets) for example JSON (or any other format you like):
{ "user": "A", "page": "client1.html" }
Now on the server side you just mark that this user/connection came from client1.html. You can add for example a custom property:
conn.source = "client1.html";
Or any other way (depending for example on framework).
You might even use a handshake for this (instead of sending JSON): when connecting to the server do for example (on the client side):
var ws = new WebSocket("ws://myserver/client1.html");
Now you just have to do the same in handshake code (client1.html is a part of URL now in handshake).
As for other question: on the server side you keep lists of all users for client1.html, client2.html, etc. The rest is obvious: you loop over a target list and send a notification to those users.
Of course there are many small details here. Like you have to remove users from lists if a connection is dead (thus you need a background task to check whether a connection is alive), etc. But that's the general idea.
I have working PHP application. It allows user create private projects and invite others in it. Now, using node.js and socket.io, I want to make real-time comments, posts etc.
What is the best architecture?
I see two solutions now.
The first is:
User sends AJAX query to PHP backend:
http://example.com/comment_add.php?text=...
comment_add.php adds
comment to database and via AMPQ (or something better?) notifies
node.js server which broadcasts comment to channel's subscribers.
The second is:
User sends AJAX query to node.js server: http://example.com:3000/comment_add
Node.js sends request to PHP backend (But how? And what about authorization?), receives response, and then broadcasts to channel's subscribers.
What is the best way? Is there another methods? How to implement this properly?
When you decided to use node.js + socket.io to make a real-time web-app, you don't need to think about PHP anymore and forget Ajax also... Socket.io will be the communication between client and server.
But yes, you can use Ajax and PHP for building websites fast, and some other functions that don't need real-time
The second way is the best method. You can use http to communicate with PHP from node.js. Authorization can be done in node.js but passing auth credentials everytime to PHP
Finally my working solution is #1.
When user establishing connection to node.js/socket.io he just send 'subscribe' message to node.js with his PHP session id. Node.js checks authorization using POST request to PHP backend and if all is OK allows user to establish connection.
Frontend sends all requests to PHP as it was before node.js.
PHP modifies some object, checks who can access modified object and sends message (via AMQP or redis pub/sub etc.) to node.js:
{
id_object: 125342,
users: [5, 23, 9882]
}
node.js then check who from listed users have active sockets and for each user sends GET request to PHP:
{
userId: 5,
id_object: 125342
}
Special PHP controller receiving this request runs query to get object with rights of given user id and then sends message to node.js with resulting answer. Node.js then via socket sends answer to user's frontend.
I faced this same question a year ago when starting my final year project at University. I realized that my project was much better suited to using Node as a standalone. Node is very good at dealing with I/O, this can be anything from a HTTP requests to a database query. Adding in a PHP based Web Server behind Node is going to add un-needed complexity. If your application needs to perform CPU intensive tasks you can quite easilly spawn 'child' node processed which perform the needed operation, and return the result to your parent node.
However out of the two you methods you have mentioned I would choose #2. Node.js can communicate with your PHP server in a number of ways, you could look at creating a unix socket connection between your PHP server and Node. If that is unavailable you could simply communicate between Node and your PHP back end using HTTP. :)
Take a look here, here is a solution to a question very similar to your own:
http://forum.kohanaframework.org/discussion/comment/57607