Currently, I have architectured our current application using nodejs & mongodb. This was done assuming that the date write costs for appengine will become too expensive than hosting nodejs and mongodb.
I need help in identifying the differences in the costs for an application which will grow to hit lets say about 50 requests/second. Each request is logged in database. The entities are really simple (id, datetime, email_address). The data is not recycled/deleted.
Use nodejs hosting provider like nodejitsu (not sure how many drones would I need for such traffic). Mongohq for db hosting. Or anyone you can prefer to cut down costs.
Use appengine for the same model with builtin datastore
Use appengine with Google Cloud SQL
Thanks.
You could put it on AppFog and get up to 512 mb ram for free.
Regarding Nodejitsu - 50 requests per second should be ok on 1 drone.
Even a very small VPS with 128mb ram could handle this load as well.
Related
I am working on a Node + MongoDB API and the API is deployed on a VM on Google Cloud Platform. Currently, the data is stored in a MongoDB instance running on the VM.
Is running a local MongoDB instance for production a good practice? And how does various cloud services providing MongoDB compare with each other? And what are some good practices to ensure the API is scalable? Also can deploying API to Kubernetes as a container offer better results as compared to VMs?
Having both server and database running in the same VM instance would always be a bad idea for a service that you want to scale. Let's think what happen if your instance could not handle the number of traffic being sent into anymore?
At first you might just do the vertical scale adding more CPU and RAM to the currently running instance and yes, that would be fine for some period of time. But it is actually more like a temporary life-extending plan since at some point, your instance will not be able to do the vertical scale anymore. And here come the horizontal scale...
New instance has to be created to handle the increasing number of traffic and how would you design the new instance for that?
The best practice is to have an instance that looks exactly the same as the first one. So the server is needed to be designed as a stateless one. In this case, your instance will be able to scale for at any number of machine.
That's why you should not have the API front end and database running in the same instance. It could not be scaled.
So how's about the database? It is quite straightforward. Just simply place it in the another VM instance! If database is still be able to handle the request, you can just share the database server among the server instances. But if there is a bottleneck on the database side, you can scale your mongoDB separately with cluster or so.
If you don't have time, you can consider using the ready-to-use service like Heroku Add-On, for example, mongolab, to managing the MongoDB instance for you.
Cheers.
We have an Azure web app for our production environment. The site is built with the Umbraco 7.2.6 CMS.
The web app Instance size is 'Large (4 Cores, 7GB memory)'
The database for this is a Standard SQL Azure S0 Level (10 DTUs).
When running this site on my local machine against the same SQL Azure database (exactly the same instance) the site is very fast.
However, on Azure the site runs painfully slowly. I cannot find any obvious reasons for this.
Does anybody have any suggestions for troubleshooting this issue?
I had exactly the same issue: Azure Web App + Azure DB = Slow DB returns.
But on the other hand if I ran the app locally on my computer and connected to the azure DB, everything would be flash quick.
So I checked my app service and location. I was using S1 located in the US with DB in Australia.
I upgraded to Premium S2
Located my App and DB is same region. Now it is >10 times faster.
I would suggest checking these two first before looking into anything else.
Change the database to S2, although it does not look taxed by the load of Umbraco it will make a big difference to performance.
Also the underlying storage of WebApps in Azure is quite slow and since Umbraco is local disc intensive then this is a factor if running a large site due to the Examine indexes.
There is a plugin replacing examine called "Azure Search for Umbraco" which will improve performance but may require a lot of rework dependant on site.
I did not expect this problem, it seems unfortunate that the way to solve this problem is to upgrade. I think I will try to utilize the In App MySql database instead.
I am building a chrome extension that needs a backend to store user information. So far, we have been using Firebase, a real-time solution that we can access from the front end. Yet we will need to store approximately 200GB of data so Firebase seems less viable for a startup budget.
We were looking into transitioning to hosting a node.js app in the cloud and then communicating with a mongodb database. We were looking into Azure for this purpose. Yet it seems the only way to do this is getting MongoLab, which is still really expensive. Isn't there a way to store a lot data in mongodb without incurring in huge costs? For some reason, the SQL databases look way cheaper, which does not make much sense to me.
some links for reference.
SQL pricing:
http://azure.microsoft.com/en-us/pricing/details/sql-database/
mongodb pricing:
https://mongolab.com/plans/pricing/
Sure you can get Mongo running in Azure. You would simply fire up a new Linux VM and install Mongo and you're off!
http://azure.microsoft.com/en-us/pricing/details/virtual-machines/#Linux
Your question hints that the biggest priority is disk space. Storage in Azure is pretty cheap. Let's imagine you get an A2 instance with 60 GB of space and you run out of space. You can easily attach new disks in Azure and the storage is really cheap.
The classic way to scale Mongo however is to use a replica set, in which case you'll need to pay for more nodes/machines as you add them.
Here's the setup:
ec2 micro instance
MySQL 5.6
Redis server
Node.js (express based app)
Nginx as reverse front-end proxy.
It's slow. Very slow. I know it's a micro instance and you get what you pay for (considering it's free).
I ended up even using a swap file for MySQL and it's so slow to the point that it's unusable. Should I spin up 2 medium instances (1 for the db/redis and one for the app server)? Keep everything on one and upgrade it to a large instance?
Also, what should I be looking for? More RAM for MySQL and more CPU for the app server? Any input would be extremely helpful (especially those that have used a similar setup in the past).
Keep in mind that EC2 micro instances throttle the cpu - the can surge a bit but if you place consistent cpu load on a micro instance it will throttle down. They are really designed for development - I've used micro instances as web servers before and have paid the price when they throttled down just when the load went up - basically ground to a halt.
As to what you should use you'll really need to assess your own needs based on a combination of benchmarking and analysis on database size, working set size, number of users etc.
That said, if you intend to scale your app at all trying to keep everything on one virtualized server tends not to work well. EC2 currently has many different instance types optimized for different usage scenarios, variously emphasizing cpu, memory, local disk or network capacity. Scaling the node.js/nginx side of your app is very different than MySQL and Redis.
Personally (and it's just my opinion) I'd start with two smalls, MySQL and Redis on one, node and nginx on the other and monitor memory, cpu and disk usage carefully. The great thing about EC2 (or any of the major cloud based virtual instance providers) is the ease with which you can experiment and move to another instance type. To facilitate that I'd definitely use an EBS volume as your database as it makes it very easy to move it later (not to mention backups using volume snapshots for backups).
I'm developing a very simple portfolio web site. It will most likely barely have any traffic. I want to use node.js and redis for the backend of the application.
Since amazon ec2 has 1 year free of micro instance, I was wondering if it will be good enough to host both redis and node.js on it.
As I said the site will be very lightweight and not accessed frequently.
Thanks!
A micro instance has been more than enough for my own site which runs nginx, wordpress, django, and a few other random things.
I'm sure it will be more than adequate for what your doing (unless it's number or IO intensive.. and if you don't know, it's probably not). And hey, it's free, so why not? AWS has a lot of cool features you can play with too!
The specs from Amazon lead me to believe so.
Obviously the redis is questionable with the allotted RAM but it should be fine for minimal traffic.
613 MiB memory
Up to 2 EC2 Compute Units (for short periodic bursts)
EBS storage only
32-bit or 64-bit platform
I/O Performance: Low
EBS-Optimized Available: No
API name: t1.micro
My understanding says that it's free for the new users, so just put your code up there and check what happens. Other way is that it only provides around 650 mb of RAM. So you should be fine at the start. Only when a lot of data gets generated by users, the RAM will become bottle neck.