When deploying a Cosmos Db database in GlobalDocumentDB mode using the Azure resource manager template there is no way to configure collections in the database during deployment that I can tell.
During a ARM template deployment is there any way to either (in order of preference)
Configure the collections in a Cosmos Db database directly in a way that I have overlooked.
Do a series of REST requests from within the template to configure the db.
Do a single REST request within the template to hit a Azure Function with a HttpTrigger that will configure the database.
You should be able to use these Templates.
Containers:
https://learn.microsoft.com/en-us/azure/templates/microsoft.documentdb/2015-04-08/databaseaccounts/apis/databases/containers
Collections:
https://learn.microsoft.com/en-us/azure/templates/microsoft.documentdb/2015-04-08/databaseaccounts/apis/databases/collections
All Microsoft.DocumentDB Resource Types:
https://learn.microsoft.com/en-us/azure/templates/microsoft.documentdb/allversions
These pages were published on 05/01/2019 & are for API version 2015-04-08.
ARM templates is used to create Azure resources, there is no way to configure collection during deploy ARM template. To configure collections, I suggest you use REST API. Before creating a collection, we need to create a database first. Following are documents which show us how to create DocumentDB databases and collections.
Create Database
Create Collection
Related
I made an ARM template which runs through an azure devops pipeline to create a new cosmos instance and put two collections inside it. I'd like to put some data inside the collections (fixed values, same every time). Everything is created in the standard way, e.g. the collections are using
"type": "Microsoft.DocumentDb/databaseAccounts/apis/databases/containers"
I think these are the relevant docs.
I haven't found mentions of automatically adding data much, but it's such an obviously useful thing I'm sure it will have been added. If I need to add another step to my pipeline to add data, that's an option too.
ARM templates are not able to insert data into Cosmos DB or any service with a data plane for many of the reasons listed in the comments and more.
If you need to both provision a Cosmos resource and then insert data into it you may want to consider creating another ARM template to deploy an Azure Data Factory resource and then invoke the pipeline using PowerShell to copy the data from Blob Storage into the Cosmos DB collection. Based upon the ARM doc you referenced above it sounds as though you are creating a MongoDB collection resource. ADF supports MongoDB so this should work very well.
You can find the ADF ARM template docs here and the ADF PowerShell docs can be found here. If you're new to using ARM to create ADF resources, I recommend first creating it using the Azure Portal, then export it and examine the properties you will need to drive with parameters or variables during deployment.
PS: I'm not sure why but this container resource path (below) you pointed to in your question should not used as it breaks a few things in ARM, namely, you cannot put a resource lock on it or use Azure Policy. Please use the latest api-version which as of this writing is 2021-04-15.
"type": "Microsoft.DocumentDb/databaseAccounts/apis/databases/containers"
Given an Azure CosmosDB DB instance that is created from the Azure portal, it is possible to create multiple databases from a shell connection with the following commands:
use someNewDbName;
db.someNewCollectionName.insert({});
With other DB providers that expose MongoDB APIs, it is possible to configure user roles on either a database or colletion level (for users that exist on the same DB instance).
For example, with self-hosted MongoDB, the db.createUser() allows the roles parameter which accepts the db option. MongoDB Atlas allows similar operations to be performed through their UI.
Is it possible to do the same with CosmosDB? Within the Azure Portal, selecting the CosmosDB, and then Access control (IAM) and then Roles leads to a list of built in roles as well as a text that says it is possible to define your own roles but no indication as to how to do that.
I am able to create custom role with following method using Powershell
This role was displayed in the list of available role under "Add Role assignment" Tab
These links might help you
https://learn.microsoft.com/en-us/azure/role-based-access-control/tutorial-custom-role-powershell
https://learn.microsoft.com/en-us/azure/role-based-access-control/custom-roles
I also tried to create users and roles for an Azure CosmosDB using the MongoDB interface and followed this documentation: https://learn.microsoft.com/en-GB/azure/cosmos-db/secure-access-to-data?tabs=using-primary-key.
It seems however that this is simply not supported by the MongoDB interface. I followed the above documentation using the role-based access control approach and eventually run into the following issue when executing the below command:
az cosmosdb sql role definition create --account-name <some-account> --resource-group <some-resource-group> --body #role-definition.json
(BadRequest) The Database Account [<some-database>] has API type
[MongoDB] which is invalid for processing SQL Role Definitions.
The above is also confirmed documentation on the (resource)token-based approach: https://learn.microsoft.com/en-us/rest/api/cosmos-db/permissions (see first line):
Azure Cosmos DB is a globally distributed multi-model database that supports the document, graph, and key-value data models. The content in this section is for managing permission resources using the SQL API via REST.
Hope that this helps.
I am developing a Saas based application which has hybrid DB architecture (Azure SQL Server and Azure Cosmos DB).
To manage SQL Server Tables, Stored procedures, triggers and functions we will create a SQLDB project (.sqlproj). Also we can generate .dacpac and deploy in the sql server.
As like SQL, we will have collections, stored procedures, triggers and functions in Azure CosmosDB.
How to manage CosmosDB collection, procedure, trigger? Is there any project templete available to manage? Suggest a solution to proceed.
Based on my experience with CosmosDb, I believe there is nothing sort of project templates available for CosmosDb. Because it is not as easy as SQL Db project.
I suggest you will have to store them as json files in local solution version control and version them accordingly.
You could write necessary programming logic to execute these scripts/cosmos DB logic using SQL API for .NET or another platform. This way you are controlling the collections, udf, triggers etc from your code, and you can version your code accordingly.
More references here: https://learn.microsoft.com/en-us/azure/cosmos-db/programming
Azure CosmosDBs can be managed through ARM templates. You can use these to version your databases/collections/etc. See Microsoft.DocumentDB resource types documentation.
I am trying to setup continuous deployment for a group of Azure databases that all share the same schema. In my situation, there are a number of dynamic databases that get created via copying and renaming a standard template. The software will make a copy of the CompanyTemplate database and rename it to Company_XXXX.
I would like to create a Task Group and/or a script in VSTS (hosted) that can query the master database, get a list of the company database names and then loop said Task Group in order to deploy the same schema and scripts to each of the Company databases that get created.
I have been Googling and testing odds and ends for days but I cannot find anything pertaining to how this can be done. Any thoughts? Is this possible?
There is no loop concept in the VSTS Build/Release environment.
There are a few workarounds that sprint to mind:
Run a powershell script and implement the logic there. Using the loop constructs in Powershell.
Run a powershell to trigger as many builds as you want using the REST API.
To begin with, I want to acknowledge that reading the answer from #jessehouwing triggered a few thoughts on my end.
As he mentions in his answer, there isn't anything that would directly do what you're asking. However, some techniques do come to mind, depending on how you want to deploy the databases.
ARM Templates -
Setup an ARM template that uses Resource Iteration to deploy multiple Azure SQL Databases. (See MS DOCS on how to do that). Configure the template to copy the schema of an existing DB to the new ones. You'll need that template DB deployed to Azure to act as the schema source. To configure the ARM template to create the new databases as a copy of the template, look at the createMode property of the SQL Database ARM template (SQL ARM Template documentation).
Run a Powershell script that queries the master DB to get the list of companies (Query DB from Powershell).
Output the results of the DB query to a VSTS variable and pass that variable into the ARM template to produce the databases.
DACPAC -
Create a DACPAC from a SQL DB Project in Visual Studio.
You can either create a DACPAC that defines just the DB schema and use the ARM template technique above to run the DACPAC for each database you need in something of a hybrid technique - or
You can create a dacpac that queries your main DB for the list of
companies and creates a database for each one based on the defined
schema. This options encapsulates the process of creating the schema
and querying the main DB for the ones to create all into a single
deployment artifact
Each option has its Pros and Cons. The ARM Template option is going to give you the most flexibility, but requires that you have a template DB in place to copy from.
The DACPAC option requires familiarity with using that technique for deploying databases and may still require an ARM template to make the process as flexible as possible. It does offer the potential to encapsulate all the DB deployment parts into a single step.
There are a fair number of variables here, but I think this should give you some options to consider that will take you in a workable direction.
I'm trying to create an ARM template that provisions a CosmosDB instance, the only documentation I've found however is for DocumentDB which I'm aware is what Cosmos used to be called. If I provision a DocumentDB cluster will that create a CosmosDB instance? If not Does anybody have any experience of provisioning a CosmosDB with an ARM template and if so, is there any reference material I can read?
Yeah, those are the same. Just use this examples repo. You are looking for any examples with documentdb or cosmosdb in the name.
Alternatively you can créate a CosmosDB instance in the portal and look at the template that is being used to créate it (under deployments in the resource group where you deployed it).