I am getting ready to create a brand new mobile application that communicates with CosmosDB and I will probably go the serverless way. The serverless way has some little disadvantages compared to the provisioned throughput (eg. only 50GB per container, no Geo-Redundancy, no Multi-region Writes, etc.).
If I need later on to convert my DB to a provisioned throughput one, can I do it somehow?
I know that I can probably use the change-feed and from that (I guess) recreate a new DB from it (provisioned throughput one) but this might open the Pandora's box especially while a mobile app connects to a specific DB.
As Gaurav mentioned, there is no way to change to Provisioned from Serverless plan once you create an account.
You will need to recreate the account with Serverless as type and follow the below ways to migrate the data,
(i) Data Migration Tool - You can easily migrate from one account to another
(ii) ChangeFeed and Restore - push the changes to the new the instance of Azure Cosmos DB
Once you are synced switch back to the new one.
Based on the documentation available here: https://learn.microsoft.com/en-us/azure/cosmos-db/serverless#using-serverless-resources, it is currently not possible to change a Cosmos DB server less account to provisioned throughput.
Related
I have a number of existing collection with manual RU provisioning which I would like to migrate to be Autopilot managed to better automatically deal with varying levels of demand.
The collections contain many GB of historical timeseries data, and I cannot have any downtime where new or historical data is not available to customers. I must also ensure no data is lost during the migration
Once a day, a new day of data is bulk uploaded to the cosmosdb collection, and the collections can be queried at any time by the customer-facing service in front of them.
For migration, I was considering the following:
1. Create new autopilot collection
2. Modify service to query both old and new collection and deduplicate any data present in both
3. Redirect data upload to new collection
4. Use ADF (Azure data factory) to copy the contents of the old collection to the new Autopilot one
5. Update service to only query the new collection
6. Drop old collection.
Is this the best migration strategy, or is there an alternative approach which would provide a better customer experience, or be less work?
While in Preview you will need to manually migrate data to AutoPilot containers. Once we GA we are planning to allow customers to seamlessly migrate containers from regular to AutoPilot throughput.
For the scenario you describe I find it easier to use ChangeFeed when I need to do a near zero downtime migration. Create a new AutoPilot configured container, then create an Azure Function using the Cosmos DB bindings to read from the source container and write to the new AutoPilot container to allow data to stay in sync.
Rewrite your consuming apps to use the new container and your bulk load scripts to write to the new container. Once that is done, deploy the changes. I like using slots for Web Apps (or whatever you choose) for zero or near zero downtime.
One thing to keep an eye on is since you are bulk loading this data, Azure Functions will likely fall far behind keeping the data in sync. You'll want to monitor to see how long that takes just so you know when you can flip the switch on the migration.
Hope that helps.
We are planning to use cosmos db single master deployment where all master data are maintained from a single region. The application is spread across various regions and we need to provide read access to the individual regions. However we would like to have filtered replication as not all regions will be interested in all data in cosmos DB. Is there any way to use selective region specific replication? I am aware that we could use Cosmos DB trigger and then have function app etc to replicate traffic but that is an overhead in terms of maintenance and monitoring. Hence would be interested to know if we can make use of any native functionality.
The built-in geo-replication mechanism is completely transparent to you. You can't see it and you can't do anything about it. There is no way to do what you described without writing something custom.
If you really want to have selected data replicated then you would need to do the following (It's a terrible solution and you should NOT go with it):
Create a main source of truth Cosmos DB account. That's "single master" that you described.
Create a few other accounts in whichever region you want.
Use a Cosmos DB trigger Azure Function or the Change Feed Processor library to listen to changes on the main account and then use your filtering logic to replicate them into the other accounts that need to use them.
Use a different connection string per application based on it's deployment environment
What's wrong with just having your data replicated across all regions though? There are no drawbacks.
Is there any way to stop a SQL Azure DB so that it doesn't charge anything towards our account? I don't want to delete it, just while in testing and it's not being used than we set it to "stopped" like we can do with websites, cloud services, and VMs.
As of 10th February, 2023, The answer is No.
They won't allow it. So the billing will continue for your Azure Database starting the day you create it. There really is no way to pause / stop billing for your Azure SQL Database.
Official Source: feedback.azure.com Please add ability to temporarily turn off/on SQL Azure server to pause billing
Microsoft's official answer appears to be "Yes, you can export your database. Delete the Azure SQL database and that will pause billing. Then when you need it you can create a new database and import your previously expored DB."
I don't believe this is acceptable as an answer for "Allow me to temporarily turn off SQL Server to save on my billing"
This is not an option today - the only choice you have is to reduce the size of the Azure SQL Database which will reduce the cost from the next hour of service. If you really don't want to pay for the DB you could backup the DB to blob storage, delete the database and then restore when required. You could orchestrate this using PowerShell or similar.
Update May 2019: There is a new Azure SQL Database "Serverless" tier coming that might meet some of the requirements around reducing costs by not billing when not in use. Official documentation is available to read.
The Azure SQL Database team is happy to announce that there is now an option that may address your request. We just announced a "serverless" option for Azure SQL DB that will pause your database when it is not in use. You can read more about the feature here:
SQL Database Serverless
The databases get backed up automatically just before a drop. so, you can just drop it when you dont need it and restore it when needed.
Restores will take some time depending on the database size and how much log you generated, so it wont be fast for large databases.
Also, there is an expiration policy on how long the backups are retained (depends on the service tier) so just watch out for that.
https://msdn.microsoft.com/en-us/library/azure/jj650016.aspx
Agree with #Shiva answer.
But if you are simply trying out SQL Server on an Azure VM, you would not want to incur charges by accidentally leaving it running over the weekend or weeks. One solution is to use Automatic Shutdown Feature.
This is now possible and the preview feature is public.
Azure SQL Database serverless
https://learn.microsoft.com/en-us/azure/sql-database/sql-database-serverless
Whereas, at this low scale (scale down requirement (pause) rather than scale up), SQL running within a VM might be a better answer for you...
As is it is not supported however there are a few work arounds. It really depends upon how long you want to stop it and how immediately you want it and the size of the database. If it is for a couple hours then it may not worth it because the billing is hourly and you may have round off issues. If days then you can drop it and restore when needed. If weeks then exporting the database and importing when needed is another option. Also you may want to check backup strategy for the edition you choose before preferring export / restore.
The other thing to note is the immediate need vs planned. If immediate, and the db is big then make sure the SLAs in place works for you
You could export the database to Azure storage and Import it when you want to re-enable it, as suggested here:
Temporarily turn off on sql
Here's what I did with my Azure database (4/20/19)
I scaled the database DTUs as low as possible while still reserving up to 250GB, which turns out to be 10 DTUs with an estimated cost of 1.50 per DTU (estimated at $15 per month). To me, that's almost as good as turning it off and on. I can scale it up to 100 DTUs when I want to get more processing and scale it down when I don't. Even at 100 DTUs for a whole month, the cost was only $75.93 last month on my test database.
NOTE: I am NOT using a VM to run a database, just the standard SQL server you get when you create a database in Azure.
Yes you can, with Azure Serverless SQL Database. While your compute resources will be suspended when the database is not in use, you'll definitely save the costs for compute resources, however the billing will continue for storage resources. You can set the inactive timeout for the database after which the compute resources will be suspended. This timeout can be as low as 1 hour.
Read this: https://azure.microsoft.com/en-in/updates/update-to-azure-sql-database-serverless-providing-even-greater-price-optimization/
Elastic PoolIf you have more than one database you can use the Elastic Pool option to bring your total cost down.
Others also mention the option to Drop your database, and rely on restore. That will also work, if you do not leave it deleted for too long...
I have an ASP.NET MVC 2 Azure application that I am trying to switch from being single tenant to multi-tenant. I have been reviewing many blogs and posts and questions here on Stack Overflow, but am still trying to wrap my head around the specifics of what's right for this particular app.
Currently the application stores some information in a SQL Azure database, as well as some other info in an Azure Storage Account. I'm considering writing the tenant provisioning code to simply create a new database for a new tenant, along with a new azure storage account. This brings me to the following question:
How will I go about testing this approach locally? As far as I can tell, the local Azure Storage Emulator only has 1 storage account. I'm not sure if I'm able to create others locally. How will I be able to test this locally? Or will it be possible?
There are many aspects to consider with multitenancy, one of which is data architecture. You also have billing, performance, security and so forth.
Regarding data architecture, let's first explore SQL storage. You have the following options available to you: add a CustomerID (or other identifyer) that your code will use to filter records, use different schema containers for different customers (each customer has its own copy of all the database objects owned by a dedicated schema in a database), linear sharding (in which each customer has its own database) and Federation (a feature of SQL Azure that offers progressive sharding based on performance and scalability needs). All these options are valid, but have different implications on performance, scalability, security, maintenance (such as backups), cost and of course database design. I couldn't tell you which one to choose based on the information you provided; some models are easier to implement than others if you already have a code base. Generally speaking a linear shard is the simplest model and provides strong customer isolation, but perhaps the most expensive of all. A schema-based separation is not too hard, but requires a good handle on security requirements and can introduce cross-customer performance issues because this approach is not shared-nothing (for customers on the same database). Finally Federations requires the use of a customer identifyer and has a few limitations; however this technology gives you more control over performance distribution and long-term scalability (because like a linear shard, Federation uses a shared-nothing architecture).
Regarding storage accounts, using different storage accounts per customer is definitively the way to go. The primary issue you will face if you don't use separate storage accounts is performance limitations, such as the maximum number of transactions per second that can be executed using a single storage account. As you are pointing out however, testing locally may be a problem; however consider this: the local emulator does not offer 100% parity with an Azure Storage Account (some functions are not supported in the emulator). So I would only use the local emulator for initial development and troubleshooting. Any serious testing, including multitenant testing, should be done using real storage accounts. This is the only way you can fully test an application.
You should consider not creating separate databases, but instead creating different object namespaces within a single SQL database. Each tenant can have their own set of tables.
Depending on how you are using storage, you can create separate storage containers or message queues per client.
Given these constraints you should be able to test locally with the storage emulator and local SQL instance.
Please let me know if you need further explanation.
I recently got a trial version of Windows Azure and wanted to know if there is any way I can deploy an application using Cassandra.
I can't speak specifically to Cassandra working or not in Azure unfortuantly. That's likely a question for that product's development team.
But the challenge you'll face with this, mySQL, or any other role hosted database is persistence. Azure Roles are in and of themselves not persistent so whatever back end store Cassandra is using would need to be placed onto soemthing like an Azure Drive (which is persisted to Azure Blob Storage). However, this would limit the scalability of the solution.
Basically, you run Cassandra as a worker role in Azure. Then, you can mount an Azure drive when a worker starts up and unmount when it shuts down.
This provides some insight re: how to use Cassandra on Azure: http://things.smarx.com/#Run Cassandra
Some help w/ Azure drives: http://azurescope.cloudapp.net/CodeSamples/cs/792ce345-256b-4230-a62f-903f79c63a67/
This should not limit your scalability at all. Just spin up another Cassandra instance whenever processing throughput or contiguous storage become an issue.
You might want to check out AppHarbor. AppHarbor is a .Net PaaS built on top of Amazon. It gives users the portability and infrastructure of Amazon and they provide a number of the rich services that Azure offers such as background tasks & load balancing plus some that it doesn't like 3rd party add-ons, dead-simple deployment and more. They already have add-ons for CouchDB, MongoDB and Redis if Cassandra got high enough on the requested features I'm sure they could set it up.