In order to predict our exploitation costs, my new associates and I would like to predict our hosting needs.
Our application would be a public one, involving increasing number of users.
We found that, for node.js applications, we basically have 2 options :
As a service, like Heroku
Take a raw server, dedicated or virtual, like OVH here in France
Specifications :
The server would be essentially a backend one, serving ressources "REST-like" over socket.io (with sails.js' implementation sails.io.js)
The usage would basically be, for each user :
Making a search : server taking a "request" (socket event), processing a reasonnable calculation (involing a few maths), returning a reasonnable number (< 1000) of "responses" (socket event), taken from a database, as json
the user would make, say, 3 requests in a raw usage)
Each user would use the application twice a day
In the background, each user would send it's location to the server, still with "REST-like" over socket, say, every minute
Question
I'd just like to know, what basically would be the process to guess the kind of server we have to purchase ? We would like to "scale as we grow" the server, but we still have to make plans, and I can't realy figure out how to predict the need for 10000 users for example.
Would this be about calculating a "per user" server performance unit (Ram, Cpu, "Dyno") and network unit (bandwidth) ?
Thank you very much =)
It's 7 months ago, but as an answer try to use Google Compute Engine, and use the auto scale function to scale use as you go. Node.js works on it and you can install any other packages you need.
Load balancing is handled for you by google as well. You pay extras, but you save a lot of time from research/developing scaling issues.
Related
Assume the Joker is a maximally sophisticated, well-equipped and malicious user of Batman's start up batmanrules.com hosted by, say, AWS infrastructure. The business logic of batmanrules.com requires that unregistered users be able to send http requests to the REST API layer of batman.com, which lead to the invocation (in one way or another) of queries against an AWS-based DB. Batman doesn't want to be constrained by DB type (it can be either SQL or noSQL).
The Joker wants to ruin batman financially by sending as many http requests as he can in order to run up Batman's AWS bill. The Joker uses all the latest tricks in the book using DDOS-like methods to send http requests from different IP addresses that target all sorts of mechanisms within batman.com's business logic.
Main Question: how does Batman prevent financial ruin while keeping his service running smoothly for his normal users?
Assume a lot of traffic is going on, how can you weed out the 'malicious' queries from the non-malicious, especially when users arent being registered? I know you can do rate-limiting against IP addresses, but cant the Joker (who is maximally sophisticated and well-equipped) find clever ways to issue requests from ever-changing IP addresses, and or to tweak the requests so that no two are exactly the same?
Note: my question focuses not on denial of service -- let's assume it's ok if the site goes down for a while -- but, rather, on Batman's financial loss. Batman has done a great job on making the architecture scale up and down with varying load, his only concern is that high loads (induced by Joker's shenanigans) entail high cost.
My instinct tells me that there is no silver bullet here, and that batman would have to build safeguards into his business logic (e.g. shut down if traffic spikes within certain parameters) AND/OR to require reCAPTCHA tokens on all non-trivial requests submitted to the REST API.
You can use AWS WAF and configure rules to block malicious users.
For example a straight forward rule would be to do a rate base blocking where if you could find its highly unlikely to get above X amount of requests concurrently from a same IP address.
For advanced use cases you can implement custom rules by analyzing the request logs with Lambda and to apply the block in WAF.
In addition, as you clearly identified it is not possible to prevent all the malicious requests. The goal should be to inspect and prevent which is an ongoing process with the right architecture in place to block requests on need basis.
I am serving my users with data fetched from an external API. Now, I don't know when this API will have new data, how would be the best approach to do that using Node, for example?
I have tried setInterval's and node-schedule to do that and got it working, but isn't it expensive for the CPU? For example, over a day I would hit this endpoint to check for new data every minute, but it could have new data every five minutes or more.
The thing is, this external API isn't ran by me. Would the only way to check for updates hitting it every minute? Is there any module that can do that in Node or any approach that fits better?
Use case 1 : Call a weather API for every city of the country and just save data to my db when it is going to rain in a given city.
Use case 2 : Send notification to the user when a given Philips Hue lamp is turned on at the time it is turned on without having to hit the endpoint to check if it is on or not.
I appreciate the time to discuss this.
If this external API has no means of notifying you when there's new data, then the only thing you can do is to "poll" it to check for new data.
You will have to decide what an "efficient design" for polling is in your specific application and given the type of data and the needs of the client (what is an acceptable latency for new data).
You also need to be sure that your service is not violating any terms of service with your polling scheme or running afoul of rate limiting that may deny you access to the server if you use it "too much".
Would the only way to check for updates hitting it every minute?
Unless the API offers some notification feature, there is no other scheme other than polling at some interval. Polling every minute is fairly quick. Do your clients really need information that is less than a minute old? Or would it really make no difference if the information was as much as 5 minutes old.
For example, in your example of weather, a client wouldn't really need temperature updates more often than probably every 10-15 minutes.
Is there any module that can do that in Node or any approach that fits better?
No. Not really. You'll probably just use some sort of timer (either repeated setTimeout() or setInterval() in a node.js app to repeatedly carry out your API operations.
Use case: Call a weather API for every city of the country and just save data to my db when it is going to rain in a given city.
Trying to pre-save every possible piece of data from an external API is probably a losing proposition. You're essentially trying to "scrape" all the data from the external API. That is likely against the terms of service and will likely also run afoul of rate limits. And, it's just not very practical.
Instead, you will probably want to fetch data upon demand (when a client requests data for Phoenix, then, and only then, do you start collecting data for Phoenix) and then once a demand for a certain type of data (temperatures in a particular city) is established, then you might want to pre-cache that data more regularly so you can notify clients of changes. If, after awhile, no clients are asking for data from Phoenix, you stop requesting updates for Phoenix any more until a client establishes demand again.
I have tried setInterval's and node-schedule to do that and got it working, but isn't it expensive for the CPU? For example, over a day I would hit this endpoint to check for new data every minute, but it could have new data every five minutes or more.
Making a remote network request is not a CPU intensive operation, even if you're doing it every minute. node.js uses non-blocking networking so most of the time during a network request, node.js isn't doing anything and isn't using the CPU at all. The only time the CPU would be briefly used is when you first send the API request and then when you receive back the result from the API call and need to process it.
Whether you really need to "poll" every minute depends upon the data and the needs of the client. I'd ask yourself if your app will work just fine if you check for new data every 5 minutes.
The method I would use to update would be contained outside of the code in a scheduled batch/powershell/bash file. In windows you can schedule tasks based upon time of day or duration since last run, so what you could do is run a simple command that will kill your application for five minutes, run npm update, and then restart your application before closing the shell.
That way you're staying out of your API and keeping code to a minimum, and if your code is inside that Node package in the update, it'll be there and ready once you make serious application changes or you need to take the server down for maintenance and updates to the low-level code.
This is a light-weight solution for you and it's a method I've used once or twice at my workplace. There are lots of options out there, and if this isn't what you're looking for I can keep looking out for you.
We are making a taksi app and driver's taxi app will be pinging server every 1-2 secs for new orders and order state updates. So if we have 10000 drivers online, that's potentially 10000 requests every second. I already chose Node.JS for it, I found that it can sustain a higher amount of requests better than PHP. I think we will be using Google App Engine, because it's infrastructure seems really safe, crash-free-proof and scalable.
Is there anything else I should know before making API for a high request amount?
I'm building out an API using Hapi.js. Some of my code is pushing small amounts of data to the API. The issue seems to be that the pusher code is swamping the API and I'm getting ECONNRESET errors -- which means messages are getting lost. I'm planning on installing a rate-limiter in the pusher code, probably node-rate-limiter (link).
The question is, what should I set that limit to? I want to max out performance for this app, so I could easily be attempting to send in thousands of messages per hour. The data just gets dumped into redis, so I doubt the code in the API will be an issue but I still need to get an idea of what kind of message rate Hapi is comfortable with. Do I need to just start with something reasonable and see how it goes? Maybe 1 message per 10 milliseconds?
Hapi = require('hapi');
server = new (Hapi.Server);
server.connection(port: config.port, routes: {
cors: {
origin: ['*']
}
});
server.route({method: 'POST', path: '/update/{id}', ...})
There is no generic answer to how many requests per second you can process. It depends upon many things in your configuration and code such as:
Type and performance of server hardware
The amount of CPU time an average request uses
Whether your requests are CPU or disk bound. If disk bounded, then it depends a lot on your database and disk performance.
Whether you implement clustering to use multiple cores (if CPU bound)
Whether you're on shared infrastructure or not
The max number of incoming connections your server is configured for
So, there is no absolute answer here that works for everyone. If you don't have some sort of design problem that is artificially limiting your concurrency, then the best way to discover what your server can actually handle is to build a test engine and test it. Find where and how it fails and either fix those issues to extend the scalability further or implement protections to avoid hitting that limit.
Note: When a public API makes rate limiting choices, it is typically done on a per-client basis and the limit is set to a value that seems to be a little above what a reasonable client would be doing. This is more to allow fair use of the server by many clients to that one single client does not consume too much of the overall resource. If issuing thousands of small requests from a single client is not considered "good practice" in using your API, then you can just pick a number that is much smaller than that for a per-client limit.
Note: You may also want to make it easier for clients by having your API let them upload multiple messages in one API request rather than lots of API requests.
We're evaluating using the toobusy module https://github.com/lloyd/node-toobusy on an app hosted on Heroku. I am not sure what an appropriate value for maxLag would be for Heroku environment. It seems like it would need a fair amount of playing around and tweaking to tune it? Anyone use this module in production and with what kind of setup (ie. dynos) and with what params?
Thanks!
I recently started running a small auto-complete RESTful service on heroku.
It serves lots of requests for each user (request is sent on each character the user types).
This service is running on two x1 dynos.
I need it to keep responding fast under load, so I'm also using toobusy. At first I've set it to max lag of 10ms but that was an ambitious goal - it denied many requests when I ran a load test.
After some tweaking - I've ended up with max lag of 40ms. It gave me a good balance between the load (amount of requests service needs to handle) and the desired response time (before denying requests).
I'm monitoring my app for denied requests due to load, so I'm able to add more dynos when needed.
I believe you'll have to play with this value and run load tests to get to the right number as it's very specific to the hardware (in your case Heroku) and what your app is doing for each request.