Imagine a situation where a user send a request to the server and then the server needs to address an outer API sending 10 request to that API, each request must be with a 1 second wait time between each of them. When each of this request is resolved and a response returned and parsed I need to send each of the response items to another outer API once each of the items is completely resolved and parsed a socket.io connection should emit the results back to the client so he won't have to wait for 10 seconds until all the requests are resolved.
How would you address this kind of situation?
Been trying to use the 'when' library, sending proxy request using 'request-promise' but still can't wrap my head around on how to construct this kind of thing.
Added a diagram:
Related
I have a structure where an application sends a POST request, the API handles the parameters and enters them into the database then another application that is supposed to run a function when the POST request was handled. The only issue I'm having is how can I make the 2nd application know when the request was handled other than running a GET request on a time interval?
Another explanation:
Client A: POSTs data
API: Handles data
Client B: GETs and displays data to user
How can I tell Client B when to do the GET request?
This is all in node.js and using express
What you need is to push data to clients when something happens server-side. This can be achieved by either server side events (which do exactly this), or websockets which create a bidirectional communication channel between server and client. Which one to choose? Check out this stack overflow post.
I am sending file to my NodeJS Express based server by HTTP POST.
After image is received by server, server starts processing. It takes few seconds and there is few steps in processing (Transform image, OCR, saving, etc).
I want to keep user informed while he waiting for server to process image with some responses ("Transforming image...","Converting...","Finishing...").
So I need some way to send data to user before actual end of processing and final response HTTP status and data.
Is this possible just by HTTP responses or I need to implement something different?
It is not possible for http to send two responses, you can build another api which the user can hit and get the status of the last request made, once you give an order to process the request, you can send the response as processing and a unique id using which the user can check the status (2nd api), and once the user hits another api to know the status, you can give him the current status as completed or in progress.
I've got an chatbot app where I want to send one message e.g. res.json("Hello") from express, then another message later e.g. res.json("How are you doing"), but want to process some code between the two.
My code seems to have some problems with this, because when I delete the first res.json() then the second one works fine and doesn't cause any problems.
Looking in my heroku logs, I get lots of gobbledy gook response from the server, with an IncomingMessage = {}, containing readableState and Server objects when I include both of these res.json() functions.
Any help would be much appreciated.
HTTP is request/response. Client sends a request, server sends ONE response. Your first res.json() is your ONE response. You can't send another response to that same request. If it's just a matter of collecting all the data before sending the one response, you can rethink your code to collect all the data before sending the one response.
But, what you appear to be looking for is "server push" where the server can send data to the client continually whenever it wants to. The usual solution for that is a webSocket connection (or socket.io which is built on top of webSocket and adds more features).
In the webSocket/socket.io architecture, the client makes a connection the server and the connection is kept open indefinitely. Then either side of the connection can send messages to the other end. This is most useful when the server wants to "push" data to the client at any time. In this case, the client establishes the connection, then the server can send data to the client over that connection at any time. The client registers a listener for incoming messages and will be notified anytime the server sends it some data.
Both webSocket and socket.io are fully supported in modern browsers and in node.js. I would personally recommend using socket.io because some of the features it adds (a messaging layer, auto-reconnect, etc...) are very useful.
To use a continuously connected socket like this, you will have to make sure your hosting infrastructure is properly configured to allow it.
res.json() always sends the response to the client immediately (calling it again will cause an error). If you need to gradually build up a response then you can progressively decorate a plain old javascript object; for example, appending items to an array. When you are done call res.json() with the constructed response.
But you should post your code so we can see what's happening.
To respond a http request, we can just use return "content" in the method function.
But for some mission-critical use cases, I would like to make sure the http
200 OK response was delivered. Any idea?
The HTTP protocol doesn't work that way. If you need an acknowledgement then you need the client to send the acknowledgement to you.
Or you should look at implementing a bi-direction socket (a sample library is socket.io) where the client can send the ACK. If it is mission critical, then don't let it be on just http, use websockets
Also you can use AJAX callbacks to gather acknowledgment. One way of creating such a solution would be UUID generated for every request and returned as a part of header
$ curl -v http://domain/url
....
response:
X-ACK-Token: 89080-3e432423-234234-23-42323
and then client make a call again
$ curl http://domain/ack/89080-3e432423-234234-23-42323
So the server would know that the given response has been acknowledge by the client. But you cannot enforce automatic ACK, it is still on the client to send it, if they don't, you have no way of knowing
PS: The UUID is not an actual UUID here, just for example shared as random number
Take a look at Microsofts asynchronous server socket.
An asynchronous server socket requires a method to begin accepting connection requests from the network, a callback method to handle the connection requests and begin receiving data from the network, and a callback method to end receiving the data (this is where your client could respond with the success or failure of the HTTP request that was made).
Example
It is not possible with HTTP, if for some reason you can't use Sockets because your implementation requires HTTP (like an API) you must acknowledge a timeout strategy with your client.
It depends on how much cases you want to handle, but for example you can state something like this:
Client generate internal identifier and send HTTP request including that "ClientID" (like a timestamp or a random number) either in the Headers or as a Body parameter.
Server responds 200 OK (or error, does not matter)
Client waits for server answer 60 seconds (you define your maximum timeout).
If it receives the response, handle it and finish.
If it does NOT receive the answer, try again after the timeout including the same "ClientID" generated in the step 1.
Server detects that the "ClientID" was already received.
Either return 409 Conflict informing that it "Already exists" and the client should know how to handle it.
Or just return 200 OK and the client never knew that it was received the first time.
Again, this depends a lot on your business / technical requirements. Because you could even get two or more consecutive loops of timeout handle.
Hope you get an idea.
as #tarun-lalwani already written is the http protocol not designed for that. What you can do is to let the app create a file and your program checks after the 200 respone the existence and the time of the remote file. This have the implication that every 200 response requires another request for the check file
I am sending requests to a route using express.js
my problem is that i am getting the requests according to the order in which they came to express.
imagine i have a create request which takes very long but i dont care to postpond it, and a get request (to the same route) that takes a very short while.
now imagine i send 3000 create requests (without waiting to responses) and then 1 get request. express is giving me the get request into the server's api only after sending in all the 3000 creations. i want to tell express "hey you, listen, if you got a request which has "get" in its body, place it in the head of the queue and not its tail". this way, express will give me the get request as the next request even that there were 2999 create request comming before it that weren't handled yet.
I tried to do a work queue in the server itself but it didn't solve the issue because the "get" job didn't reach my api at all until I handled many many create requests.
can i do that? if so, how?
Thanks for any help.