I am developing an app that uses long-polling for instant notifications about changes made to base.
App structure is as follows:
there is a questions base and admins are answering them in real time. I have a actions table where actions are stored with timestamps. So every admins browser is polling method that queries database for those actions and if there appears a new action from last check it issues a question update with ajax.
I came to 2 questions i cant find an answer to so i came here.
I have multiple clients that are polling one database. With help of long-polling technique i was able to reduce the latency of process but the server load is still same since its asking database for changes every 2 seconds. Is there a way to reduce amount of database queries somehow because at the moment it seems wrong that lets say 10 clients are asking essentially same information each every 2 seconds. maybe there is technique to reduce server load.
Whether there is a solution to first question or not either way i need to somehow calculate server needs for this kind of app as i might need to increase amount of administrators using it. How could i do that.
Related
Let me start with a brief explanation of this question. I've been recently getting started into web development. I know the bits and bytes of modern MVC architectures and have already been using it. But now that I am planning to build a Dashboard, I am confused about how to do it.
The traditional flow of data is, it is loaded from database at the backend, processed a little and sent in a response to the frontend. This is what I have worked on till now.
But talking about dashboards, assume that the data that should be shown on the dashboard requires too much processing. Like suppose the dashboard shows how much the person has spent/earned till now.
Now, adding some background context. If the application was about showing how much the person has spent at an online shopping site till now, it might not need too much processing. Loading a few hundreds of records might be okay (maybe!).
But if the dashboard is about showing how much an intraday trader has spent/earned till now (since he/she joined) would be too heavy. In this case, loading tens of thousands of records and calculating (adding/subtracting) will result in load on the server.
In this case, I am thinking about having a separate table for reports which is updated as and when necessary (for ex: when a new trade is executed by the user). And then simply reading and sending the data to frontend. Also, a bit of processing maybe done if needed, but all the heavy work is already done.
These two examples can be classified into two categories:
Processing in real-time
Processing ahead of time
Now, the question I have here is, "Should I always use the 2nd approach?" I have this question because if the processing is too less, then creating separate tables for reports is just a waste of storage. So, there are trade-offs that I can think of.
Now, coming to the second part of this question. This part refers to how should I be handling data changes at frontend. Three possible ways are:
Periodic changes (okay if the data updates are less frequent)
Instant / Real-Time changes (using something like data streaming maybe?)
Instant / Real-Time changes, but only sending the updates (new changes that took place) and not whole data to frontend
Now, coming to what I think about this part. If the data changes are less frequent i.e. once in a day, it's okay to go for the first approach. But I am still unaware about the 2nd and 3rd one. Also, should I use data streaming and can anyone provide some insights of how streaming differs from a normal request-response cycle? I think this might be related to websockets or something similar but I am unaware.
PS: I am currently using Django as my backend, and am planning to use React/Vue or similar framework as my frontend. (The answer will be independent of this, but just for information)
With modern front-end technologies, it doesn't really matter if you use pulling data from the server (periodic update) or pushing data to the front-end (instant/real-time). For angular and React you should look at Observables, using this on the front-end both options are treated exactly the same.
In an event-driven front-end, it doesn't matter how you get the data you will write your program to handle the event of "when my data arrives, I will do XYZ"
Generally speaking, it would be better to break your next question down in to 1 or 2 very specific questions. Instead of asking a very broad question with tags "front-end", "back-end" and "everything-in-the middle", you could ask several specific questions focused on each aspect.
For small app they are no problem.
But for apps with traffic you can hit limits easily.
Http protocol is req-res driven. Just because your backend is stuck with limit, you can't really wait to send respond back until rate limit allows you to resume making your api calls.
What do you do?
I can think of several scenarios:
Wait it out: while it sucks, but sometimes it's easy fix, as you don't need to do anything.
Queue it: this a lot of work oppose to making just api call. This requires that first you store it in database, then have background task go through database and do the task. Also user would be told "it is processing" not "it's done"
Use lot of apis: very hacky... and lot of trouble to manage. Say you are using amazon, now you would have to create, verify, validate like 10 accounts. Not even possible for where you need to verify with say domain name. Since amazon would know account abc already owns it.
To expand on what your queueing options are:
Unless you can design the problem of hitting this rate limit out of existence as #Hammerbot walks through, I would go with some implementation of queue. The solution can scale in complexity and robustness according to what loads you're facing and how many rate limited APIs you're dealing with.
Recommended
You use some library to take care of this for you. Node-rate-limiter looks promising. It still appears you would have to worry about how you handle your user interaction (make them wait, write to a db/cache-service and notify them later).
"Simplest case" - not recommended
You can implement a minimally functioning queue and back it with a database or cache. I've done this before and it was fine, initially. Just remember you'll run into needing to implement your own retry logic, will have to worry about things like queue starvation **. Basically, the caveats of rolling your own < insert thing whose implementation someone already worried about > should be taken into consideration.
**(e.g. your calls keep failing for some reason and all of a sudden your background process is endlessly retrying large numbers of failing queue work elements and your app runs out of memory).
Complex case:
You have a bunch of API calls that all get rate-limited and those calls are all made at volumes that make you start considering decoupling your architecture so that your user-facing app doesn't have to worry about handling this asynchronous background processing.
High-level architecture:
Your user-facing server pushes work units of different type onto different queues. Each of these queues corresponds to a differently rate-limited processing (e.g. 10 queries per hour, 1000 queries per day). You then have a "rate-limit service" that acts as a gate to consuming work units off the different queues. Horizontally distributed workers then only consume items from the queues if and only if the rate limit service says they can. The results of these workers could then be written to a database and you could have some background process to then notify your users of the result of the asynchronous work you had to perform.
Of course, in this case you're wading into a whole world of infrastructure concerns.
For further reading, you could use Lyft's rate-limiting service (which I think implements the token bucket algorithm to handle rate limiting). You could use Amazon's simple queueing service for the queues and Amazon lambda as the queue consumers.
There are two reasons why rate limits may cause you problems.
Chronic: (that is, a sustained situation). You are hitting rate limits because your sustained demand exceeds your allowance.
In this case, consider a local cache, so you don't ask for the same thing twice. Hopefully the API you are using has a reliable "last-modified" date so you can detect when your cache is stale. With this approach, your API calling is to refresh your cache, and you serve requests from your cache.
If that can't help, you need higher rate limits
Acute: your application makes bursts of calls that exceed the rate limit, but on average your demand is under the limit. So you have a short term problem. I have settled on a brute-force solution for this ("shoot first, ask permission later"). I burst until I hit the rate limit, then I use retry logic, which is easy as my preferred tool is python, which supports this easily. The returned error is trapped and retry handling takes over. I think every mature library would have something like this.
https://urllib3.readthedocs.io/en/latest/reference/urllib3.util.html
The default retry logic is to backoff in increasingly big steps of time.
This has a starvation risk, I think. That is, if there are multiple clients using the same API, they share the same rate limit as a pool. On your nth retry, your backoff may be so long that newer clients with shorter backoff times are stealing your slots ... by the time your long backoff time expires, the rate limit has already been consumed by a younger competitor, so you now retry even longer, making the problem worse,although at the limit, this just means the same as the chronic situation: the real problem is your total rate limit is insufficient, but you might not be sharing fairly among jobs due to starvation. An improvement is to provide a less naive algorithm, it's the same locking problem that you do in computer science (introducing randomisation is a big improvement). Once again, a mature library is aware of this and should help with built-in retry options.
I think that this depends on which API you want to call and for what data.
For example, Facebook limits their API call to 200 requests per hour and per user. So if your app grows, and you are using their OAuth implementation correctly, you shouldn't be limited here.
Now, what data do you need? Do you really need to make all these calls? Is the information you call somewhat storable on any of your server?
Let's imagine that you need to display an Instagram feed on a website. So at each visitor request, you reach Instagram to get the pictures you need. And when your app grows, you reach the API limit because you have more visitors than what the Instagram API allows. In this case, you should definitely store the data on your server once per hour, and let your users reach your database rather than Instagram's one.
Now let's say that you need specific information for every user at each request. Isn't it possible to let that user handle his connection to the API's? Either by implementing the OAuth 2 flow of the API or by asking the user their API informations (not very secure I think...)?
Finally, if you really can't change the way you are working now, I don't see any other options that the ones you listed here.
EDIT: And Finally, as #Eric Stein stated in his comment, some APIs allow you to rise your API limit by paying (a lot of SaaS do that), so if your app grows, you should afford to pay for those services (they are delivering value to you, it's fair to pay them back)
We are currently in the process of organising a student conference.
The issue is that we offer several different events at the same time over the course of a week. The conference runs the whole day.
It's currently been operating on a first come, first served basis, however this has led to dramatic problems in the past, namely the server crashing almost immediately, as 1000+ students all try to get the best events as quickly as they can.
Is anyone aware of a way to best handle this so that each user has the best chance of enrolling in the events they wish to attend, firstly without the server crashing and secondly with people registering for events which have a maximum capacity, all within a few minutes? Perhaps somehow staggering the registration process or something similar?
I'm aware this is a very broad question, however I'm not sure where to look to when trying to solve this problem...
Broad questions have equally broad answers. There are broadly two ways to handle it
Write more performant code so that a single server can handle the load.
Optimize backend code; cache data; minimize DB queries; optimize DB queries; optimize third party calls; consider storing intermediate things in memory; make judicious use of transactions trading off consistency with performance if possible; partition DB.
Horizontally scale - deploy multiple servers. Put a load balancer in front of your multiple front end servers. Horizontally scale DB by introducing multiple read slaves.
There are no quick fixes. It all starts with analysis first - which parts of your code are taking most time and most resources and then systematically attacking them.
Some quick fixes are possible; e.g. search results may be cached. The cache might be stale; so there would be situations where the search page shows that there are seats available, when in reality the event is full. You handle such cases on registration time. For caching web pages use a caching proxy.
this is not so much a "how to" question than a concern I have for the security of my website...
This new website I am building is the first attempt at MySQLi, I am slowly picking it up as I go, however I have noticed a few security issues along the way which concern me and as I don't know much about the new system, I was hoping someone may hold the answers.
1 -- I've noticed a new MySQLi database connection ideally must be made on each separate page where a database connection occurs, rather than having a single db.php page and including that at the top of each page with include(db.php);- this doesn't seem to work well with MySQLi. Surely though that exposes your database details including your password to every page available for people to download with programs? Security hole or advantage?
2 -- Also in relation to the question above, with all these new connections being run every few minutes or so when the site is busy, how will your database not be overrun or overload and crash?
Like I said, I'm pretty new to MySQLi so these probably seem pretty obvious to you, but to a learner like me, I am yet to know the answer.
Thanks for any responsible replies.
Aidan.
Surely though that exposes your database details including your
password to every page available for people to download with programs?
Users don't see your PHP source code in their browsers. The PHP is executed on the server, and returns whatever it wants to the browser.
Also in relation to the question above, with all these new connections
being run every few minutes or so when the site is busy, how will your
database not be overrun or overload and crash?
For a busy site it would be more like "hundreds of times per second" as opposed to "every few minutes".
The connection is used only for as long as it takes to read the data from the database into some in-memory structure. This is usually a couple of milliseconds. The connection is then closed or sent back to the connection pool.
I'm not sure if I've quite grokked node.js yet, but I really want to implement it, cause what I do understand is pretty friggen sweet.
I've got a mobile application that uses an API from a third party. Users typically open it up to see if anything is new. It occurred to me that so long as I respect the third party API's polling limits (and other restrictions) I could simulate a push based system and allow the user to be notified once something is new.
Basically implement all the API polling from a Node.js server on some sort of interval, and make the mobile app point to my Node.js server instead of the end point API.
I figure that this will be good for a number of reasons:
Takes load off the phone's data usage (since I can cache things on both the phone and the server). This is a huge win for users who have a pay-per-byte data plan
Allows a central location for storing / accessing all the data
Lets me do some optimization on the server side (if two users happen to subscribe to the same feed I can get that in one request.
I figure this could be bad for a number of reasons:
If my server goes down, then all my apps die. By acting as a go-between my Node.js implementation may very well introduce a higher number of fail points.
When the third party releases additions to the API, it requires me to implement the changes in two places, instead of one.
My question is this: In general, is this good practice? If not, why?
Your proxy idea is fine, in that it:
converts poll to push
insulates the client from API changes
allows for some optimizations
I feel only #1 is really important.