I made an ARM template which runs through an azure devops pipeline to create a new cosmos instance and put two collections inside it. I'd like to put some data inside the collections (fixed values, same every time). Everything is created in the standard way, e.g. the collections are using
"type": "Microsoft.DocumentDb/databaseAccounts/apis/databases/containers"
I think these are the relevant docs.
I haven't found mentions of automatically adding data much, but it's such an obviously useful thing I'm sure it will have been added. If I need to add another step to my pipeline to add data, that's an option too.
ARM templates are not able to insert data into Cosmos DB or any service with a data plane for many of the reasons listed in the comments and more.
If you need to both provision a Cosmos resource and then insert data into it you may want to consider creating another ARM template to deploy an Azure Data Factory resource and then invoke the pipeline using PowerShell to copy the data from Blob Storage into the Cosmos DB collection. Based upon the ARM doc you referenced above it sounds as though you are creating a MongoDB collection resource. ADF supports MongoDB so this should work very well.
You can find the ADF ARM template docs here and the ADF PowerShell docs can be found here. If you're new to using ARM to create ADF resources, I recommend first creating it using the Azure Portal, then export it and examine the properties you will need to drive with parameters or variables during deployment.
PS: I'm not sure why but this container resource path (below) you pointed to in your question should not used as it breaks a few things in ARM, namely, you cannot put a resource lock on it or use Azure Policy. Please use the latest api-version which as of this writing is 2021-04-15.
"type": "Microsoft.DocumentDb/databaseAccounts/apis/databases/containers"
Related
I can't seem to find documentation saying this can be done? But neither saying that this can't be done. So wondering if anyone else has had this idea/query in past.
I have ARM template to generate our new data lake and it's creating storage account and container. It'd have been great to create folder structure also using the template. Or maybe the directories don't count as a 'resource'?
ARM api does not support that: https://learn.microsoft.com/en-us/rest/api/storagerp/blobcontainers/create
you can only work on containers\file shares\tables and queues. You cannot create object inside those. So its not possible with ARM Templates. They can only operate against ARM apis.
you can use deploymentScript resource as a workaround
I am trying to setup continuous deployment for a group of Azure databases that all share the same schema. In my situation, there are a number of dynamic databases that get created via copying and renaming a standard template. The software will make a copy of the CompanyTemplate database and rename it to Company_XXXX.
I would like to create a Task Group and/or a script in VSTS (hosted) that can query the master database, get a list of the company database names and then loop said Task Group in order to deploy the same schema and scripts to each of the Company databases that get created.
I have been Googling and testing odds and ends for days but I cannot find anything pertaining to how this can be done. Any thoughts? Is this possible?
There is no loop concept in the VSTS Build/Release environment.
There are a few workarounds that sprint to mind:
Run a powershell script and implement the logic there. Using the loop constructs in Powershell.
Run a powershell to trigger as many builds as you want using the REST API.
To begin with, I want to acknowledge that reading the answer from #jessehouwing triggered a few thoughts on my end.
As he mentions in his answer, there isn't anything that would directly do what you're asking. However, some techniques do come to mind, depending on how you want to deploy the databases.
ARM Templates -
Setup an ARM template that uses Resource Iteration to deploy multiple Azure SQL Databases. (See MS DOCS on how to do that). Configure the template to copy the schema of an existing DB to the new ones. You'll need that template DB deployed to Azure to act as the schema source. To configure the ARM template to create the new databases as a copy of the template, look at the createMode property of the SQL Database ARM template (SQL ARM Template documentation).
Run a Powershell script that queries the master DB to get the list of companies (Query DB from Powershell).
Output the results of the DB query to a VSTS variable and pass that variable into the ARM template to produce the databases.
DACPAC -
Create a DACPAC from a SQL DB Project in Visual Studio.
You can either create a DACPAC that defines just the DB schema and use the ARM template technique above to run the DACPAC for each database you need in something of a hybrid technique - or
You can create a dacpac that queries your main DB for the list of
companies and creates a database for each one based on the defined
schema. This options encapsulates the process of creating the schema
and querying the main DB for the ones to create all into a single
deployment artifact
Each option has its Pros and Cons. The ARM Template option is going to give you the most flexibility, but requires that you have a template DB in place to copy from.
The DACPAC option requires familiarity with using that technique for deploying databases and may still require an ARM template to make the process as flexible as possible. It does offer the potential to encapsulate all the DB deployment parts into a single step.
There are a fair number of variables here, but I think this should give you some options to consider that will take you in a workable direction.
I'm trying to create an ARM template that provisions a CosmosDB instance, the only documentation I've found however is for DocumentDB which I'm aware is what Cosmos used to be called. If I provision a DocumentDB cluster will that create a CosmosDB instance? If not Does anybody have any experience of provisioning a CosmosDB with an ARM template and if so, is there any reference material I can read?
Yeah, those are the same. Just use this examples repo. You are looking for any examples with documentdb or cosmosdb in the name.
Alternatively you can créate a CosmosDB instance in the portal and look at the template that is being used to créate it (under deployments in the resource group where you deployed it).
When deploying a Cosmos Db database in GlobalDocumentDB mode using the Azure resource manager template there is no way to configure collections in the database during deployment that I can tell.
During a ARM template deployment is there any way to either (in order of preference)
Configure the collections in a Cosmos Db database directly in a way that I have overlooked.
Do a series of REST requests from within the template to configure the db.
Do a single REST request within the template to hit a Azure Function with a HttpTrigger that will configure the database.
You should be able to use these Templates.
Containers:
https://learn.microsoft.com/en-us/azure/templates/microsoft.documentdb/2015-04-08/databaseaccounts/apis/databases/containers
Collections:
https://learn.microsoft.com/en-us/azure/templates/microsoft.documentdb/2015-04-08/databaseaccounts/apis/databases/collections
All Microsoft.DocumentDB Resource Types:
https://learn.microsoft.com/en-us/azure/templates/microsoft.documentdb/allversions
These pages were published on 05/01/2019 & are for API version 2015-04-08.
ARM templates is used to create Azure resources, there is no way to configure collection during deploy ARM template. To configure collections, I suggest you use REST API. Before creating a collection, we need to create a database first. Following are documents which show us how to create DocumentDB databases and collections.
Create Database
Create Collection
I am trying to create an ARM templates for Azure SQL DB deployment. I started by exporting template for an existing Azure sql database which was created and configured from portal. However I see quite a few fields which look a bit unfamiliar to me on usage - examples provided below:
a. kind
b. serviceLevelObjective
c. currentServiceObjectiveId
d. requestedServiceObjectiveId
e. containmentState
f. readScale
Is there any place where I can find information on what each of these properties/keys means and what are the valid values against these so that I know how to use these?
Your best bet is Azure Rest API Reference it has got examples and definitions for parameters.
Also, you don't need all of those, you can just use a "regular" way of creating Azure SQL stuff defined here.
Also, MSDN reference: https://msdn.microsoft.com/en-us/library/azure/mt163685.aspx