I have situation in my NodeJS API where I need to handle complex request which usually takes about few seconds to process. Problem is when I'm doing load tests on this API call I'm starting to receive back Timeouts (but requests still been processed in background). How and what is best option to handle this situation? Should I increase timeout or do something else? Where I need to do that, on NodeJS side or maybe on Nginx side because I use Nginx as reverse proxy?
Related
I have a Node.js project in which the front end needs to request some data from the back end. Since it only has to do so once per session, I was thinking of using web requests instead of socket.io because I don't need a continuous connection between the front end and the back end all the time.
So my question is: How much, if at all, will the efficiency of my project increase if I use web requests instead of socket.io?
If you're making a one-time request from client to server, there is no reason to use a continuous socket.io connection or to create a socket.io connection, use it for one message and then disconnect it.
It will certainly be simpler to just use a single http request to get your data.
How much, if at all, will the efficiency of my project increase?
The main difference would be at high scale your server wouldn't need to handle lots of simultaneous socket.io connections. At small scale, you probably won't notice much of difference either way. The main reason for choosing an http request is that it's the simpler and proper architecture for making a single request from client to server. A socket.io connection has it's uses for different circumstances.
Best way to respond restfully to a time consuming calculation (Node.js with Express)
I'm building a REST API using Node.js with Express.
Now I have to design the response to a time consuming calculation on the server side to a http request witch is ending up in a zip file with a few megabytes.
The calculation is done in a few seconds at the moment. But perhaps, if data grows up, it can take minutes sometime. So I think it's not the best way to wait in Express till data is ready and then res.sendfile() it back to the client. But what is the best solution?
Should I implement a delayed response with long polling? Like http-delayed-response
What I don't like here is that I immediately have to report HTTP 202. If the generation of the ZIP file fails, I have to deal with this in the response and cannot report it via HTTP status code. Also I cannot respond the file directly - as far as I know.
Or something with job ids and states to each job? Like express-delayed-response
In this way my API is not stateless anymore and I have to build a status handler.
Are there better and more established solutions to this problem?
I want to restrict HTTP server to processing at most 1 request at a time. The use case is to enable linear-inspection of the logs when troubleshooting.
In principle, I could implement the queueing logic in the requestListener. This would simply require to know when the last request has been handled.
However, I wonder if there is a lower-level approach, e.g. delaying the connection response.
We have a C# Web API server and a Node Express server. We make hundreds of requests from the C# server to a route on the Node server. The route on the Node server does intensive work and often doesn't return for 6-8 seconds.
Making hundreds of these requests simultaneously seems to cause the Node server to fail. Errors in the Node server output include either socket hang up or ECONNRESET. The error from the C# side says
No connection could be made because the target machine actively refused it.
This error occurs after processing an unpredictable number of the requests, which leads me to think it is simply overloading the server. Using a Thread.Sleep(500) on the C# side allows us to get through more requests, and fiddling with the wait there leads to more or less success, but thread sleeping is rarely if ever the right answer, and I think this case is no exception.
Are we simply putting too much stress on the Node server? Can this only be solved with Load Balancing or some form of clustering? If there is an another alternative, what might it look like?
One path I'm starting to explore is the node-toobusy module. If I return a 503 though, what should be the process in the following code? Should I Thread.Sleep and then re-submit the request?
It sounds like your node.js server is getting overloaded.
The route on the Node server does intensive work and often doesn't return for 6-8 seconds.
This is a bad smell - if your node process is doing intense computation, it will halt the event loop until that computation is completed, and won't be able to handle any other requests. You should probably have it doing that computation in a worker process, which will run on another cpu core if available. cluster is the node builtin module that lets you do that, so I'll point you there.
One path I'm starting to explore is the node-toobusy module. If I return a 503 though, what should be the process in the following code? Should I Thread.Sleep and then re-submit the request?
That depends on your application and your expected load. You may want to refresh once or twice if it's likely that things will cool down enough during that time, but for your API you probably just want to return a 503 in C# too - better to let the client know the server's too busy and let them make their own decision then to keep refreshing on its behalf.
The question is in the title. In another words, if Nginx works as the same event-driven async IO model of node.js, why doesn't it requires writing async style code? I know, Nginx is NOT actually executing any code, rather proxying them to who can. Then why doesn't node do so? Are we missing anything in the current Ngninx way? Or, gaining anything more from node (apart from the pain of writing async codes)?
Ps.
To be more specific, how different is Nginx+php-fpm or Nginx+wsgi+python/ruby from node alone regarding performance or utilizing computing resource that node claims? Couldn't node just use existing FastCGI models, be a sync style JavaScript interpreter and let webserver do its async job?
Cross-posted from NodeJS google groups:
Okay i'll try my best to answer your question:
Nginx is a web server that only proxies requests. Now if you take the example of Nginx+php+fpm or Nginx+wsgi+ruby you are having an asynchronous, evented web server sitting in front of webserver that is executing synchronously. So Nginx will accept() as many connections as possible and all of them would be queued. The requests from Nginx to your backend synchronous server would be asynchronous. But your backend synchronous server which also does accept() is not queuing any connections. It can serve only one request at a time (considering you are single threaded) and multiple requests at a time (prefork/fork(slow)/multithreaded -> has its own drawbacks like thread creation time(can be avoided with thread-pools but PITA to implement), context switches, thread deadlocks, number of connections accept()ed can never be greater than number of threads etc)
Imagine you have 2 routes to your backend server that Nginx is hitting:
/404, /login.
If the /login route is doing a lot of I/O and if another request is made to /404, the rendering of the /404 page will depend on the completion of /login's request (because the process is blocked). So basically the response to any request will depend on the request that takes the longest time to do I/O. So even though Nginx is async and evented its response time for any request will depend entirely on that one request that takes the longest time to finish (culprit: the synchronous backend server).
Now if you take the example of NodeJS, everything is asynchronous and evented. Be it File/Network I/O etc. So nothing blocks the process. So taking the previous example, even if /login route is doing a lot of I/O its all asynchronous and /404 page is rendered immediately.
My explanation is quite rudimentary. But I think it should give you more clarity.
nginx is a simple static HTTP and proxy server. Node.js is a full-featured application platform.
Why would you not expect the more specialised application to have abstracted away all the internal workings that you don't need to control directly?
Edit:
Your PS is pretty similar to this question, and is concerned specifically with using Node.JS as an HTTP server. Bear in mind that v0.4.12 had just been released when that question was closed - v0.8.5 is the latest stable release at the moment. The key point anyway is it depends what you're trying to achieve.
This blog post describes a Node.JS-based set-up achieving 250k concurrent connections on a single server. A quick google search shows people attempting similar with nginx+php struggling to reach 100k with far more hardware resources available.