I am building a chrome extension that needs a backend to store user information. So far, we have been using Firebase, a real-time solution that we can access from the front end. Yet we will need to store approximately 200GB of data so Firebase seems less viable for a startup budget.
We were looking into transitioning to hosting a node.js app in the cloud and then communicating with a mongodb database. We were looking into Azure for this purpose. Yet it seems the only way to do this is getting MongoLab, which is still really expensive. Isn't there a way to store a lot data in mongodb without incurring in huge costs? For some reason, the SQL databases look way cheaper, which does not make much sense to me.
some links for reference.
SQL pricing:
http://azure.microsoft.com/en-us/pricing/details/sql-database/
mongodb pricing:
https://mongolab.com/plans/pricing/
Sure you can get Mongo running in Azure. You would simply fire up a new Linux VM and install Mongo and you're off!
http://azure.microsoft.com/en-us/pricing/details/virtual-machines/#Linux
Your question hints that the biggest priority is disk space. Storage in Azure is pretty cheap. Let's imagine you get an A2 instance with 60 GB of space and you run out of space. You can easily attach new disks in Azure and the storage is really cheap.
The classic way to scale Mongo however is to use a replica set, in which case you'll need to pay for more nodes/machines as you add them.
Related
I am working on a Node + MongoDB API and the API is deployed on a VM on Google Cloud Platform. Currently, the data is stored in a MongoDB instance running on the VM.
Is running a local MongoDB instance for production a good practice? And how does various cloud services providing MongoDB compare with each other? And what are some good practices to ensure the API is scalable? Also can deploying API to Kubernetes as a container offer better results as compared to VMs?
Having both server and database running in the same VM instance would always be a bad idea for a service that you want to scale. Let's think what happen if your instance could not handle the number of traffic being sent into anymore?
At first you might just do the vertical scale adding more CPU and RAM to the currently running instance and yes, that would be fine for some period of time. But it is actually more like a temporary life-extending plan since at some point, your instance will not be able to do the vertical scale anymore. And here come the horizontal scale...
New instance has to be created to handle the increasing number of traffic and how would you design the new instance for that?
The best practice is to have an instance that looks exactly the same as the first one. So the server is needed to be designed as a stateless one. In this case, your instance will be able to scale for at any number of machine.
That's why you should not have the API front end and database running in the same instance. It could not be scaled.
So how's about the database? It is quite straightforward. Just simply place it in the another VM instance! If database is still be able to handle the request, you can just share the database server among the server instances. But if there is a bottleneck on the database side, you can scale your mongoDB separately with cluster or so.
If you don't have time, you can consider using the ready-to-use service like Heroku Add-On, for example, mongolab, to managing the MongoDB instance for you.
Cheers.
I've read all of the documentation about App Service Local Cache but I am struggling to see how it is useful. It claims to basically create a read-only copy of your Site directory, which for an MVC app is basically the whole app. But I can't find any information about use cases or why you'd want to do this.
I ask because it's been suggested that we move to implementing it, and I am trying to work out why we should do this.
I can see advantages if you do lots of reading/writing to disk but hardly any apps do that these days, we just use the database for everything, and logging goes directly to OMS.
Am I missing something major about this feature? To make my question non-vague, does this feature offer something useful for a simple MVC website that displays data from a database and writes back to the database?
Even if your app doesn't perform a large amount of I/O operations, you can still benefit from using App Service Local Cache due to:
Quicker app restarts (since files are local, latency to the shared network drive is removed). Helpful for app settings updates.
Less application downtime if your app loses connectivity with the shared network drive (which causes restarts), which can happen during Azure update/patch operations on the underlying VM
More are discussed in the Channel 9 video for Local Cache https://channel9.msdn.com/Shows/Cloud+Cover/Episode-201-Azure-Web-App-Local-Cache-with-Cory-Fowler
We have an Azure web app for our production environment. The site is built with the Umbraco 7.2.6 CMS.
The web app Instance size is 'Large (4 Cores, 7GB memory)'
The database for this is a Standard SQL Azure S0 Level (10 DTUs).
When running this site on my local machine against the same SQL Azure database (exactly the same instance) the site is very fast.
However, on Azure the site runs painfully slowly. I cannot find any obvious reasons for this.
Does anybody have any suggestions for troubleshooting this issue?
I had exactly the same issue: Azure Web App + Azure DB = Slow DB returns.
But on the other hand if I ran the app locally on my computer and connected to the azure DB, everything would be flash quick.
So I checked my app service and location. I was using S1 located in the US with DB in Australia.
I upgraded to Premium S2
Located my App and DB is same region. Now it is >10 times faster.
I would suggest checking these two first before looking into anything else.
Change the database to S2, although it does not look taxed by the load of Umbraco it will make a big difference to performance.
Also the underlying storage of WebApps in Azure is quite slow and since Umbraco is local disc intensive then this is a factor if running a large site due to the Examine indexes.
There is a plugin replacing examine called "Azure Search for Umbraco" which will improve performance but may require a lot of rework dependant on site.
I did not expect this problem, it seems unfortunate that the way to solve this problem is to upgrade. I think I will try to utilize the In App MySql database instead.
We want to save a few bucks and share our 1GB dedicated Azure Redis Cache between Development, Test, QA and maybe even production.
Is there a better way than prefixing all keys with an environment string like "Dev_[key]", "Test_[key]" etc.
We are using the StackExchange Redis client for .NET.
PS: We tried using the cheap 250GB (Shared infrastructure), but had very slow performance. Read operations were consistent between 600-800ms... without any load (for a ~300KB object). Upgrading to dedicated 1GB services changed that to 30-40ms. See more here: StackExchange.Redis with Azure Redis is unusably slow or throws timeout errors
One approach is to use multiple Redis databases. I'm assuming this is available in your environment :)
Some advantages over prefixing your keys might be:
data is kept separate, you can flushdb in test and not touch the production data
keys are smaller and consume less memory
The main disadvantage would be not taking advantage of multiple cores, like you could do if you ran multiple instances of Redis on the same server. Obviously not an issue in this case. Also note that this feature is not deprecated, like one of the answers suggests.
Another thing I've seen people complain about is that databases are numbered, they don't have meaningful names. Some people create a hash in database 0 that maps each number to a name.
Here is another idea to save some bucks: use separate Redis cache machines for each environment - so no problems with the keys, but stop them when you don't use them, like in the weekend and during nights. Probably more than 50% of the time you are not using them. I think it would be easy to start and stop them with some PowerShell script, we are using AWS and here it is possible.
Now from what I see the Redis persistence in Azure is not enabled, but they started working on it http://feedback.azure.com/forums/169382-cache/status/191763 - it would be nice to do a RDB snapshot before stopping and then on start to load it. So if you need to save some values and reload them on start you should do it manually (with your own service).
Currently, I have architectured our current application using nodejs & mongodb. This was done assuming that the date write costs for appengine will become too expensive than hosting nodejs and mongodb.
I need help in identifying the differences in the costs for an application which will grow to hit lets say about 50 requests/second. Each request is logged in database. The entities are really simple (id, datetime, email_address). The data is not recycled/deleted.
Use nodejs hosting provider like nodejitsu (not sure how many drones would I need for such traffic). Mongohq for db hosting. Or anyone you can prefer to cut down costs.
Use appengine for the same model with builtin datastore
Use appengine with Google Cloud SQL
Thanks.
You could put it on AppFog and get up to 512 mb ram for free.
Regarding Nodejitsu - 50 requests per second should be ok on 1 drone.
Even a very small VPS with 128mb ram could handle this load as well.