Azure - restore CosmosDB account using Terraform - azure

I am trying to create a new Azure CosmosDB account in terraform account using:
create_mode = "Restore"
Basically I am trying to restore from an existing DB, and the code needs another input attribute, of the source DB:
"source_cosmosdb_account_id" = "/subscriptions/33f91226-e87e-4cdf67a1dae4e/providers/Microsoft.DocumentDB/locations/westeu/restorableDatabaseAccounts/test-source-db-name"
I am following the format indicated by the docs:
The example is /subscriptions/{subscriptionId}/providers/Microsoft.DocumentDB/locations/{location}/restorableDatabaseAccounts/{restorableDatabaseAccountName}
However when I apply the code, I get the following error:
Code="BadRequest" Message="Failed to parse uri
/subscriptions/33f91226-e87e-4ca1dae4e/providers/Microsoft.DocumentDB/locations/westeu/restorableDatabaseAccounts/test-source-db-name
The issue seems to be the way I write the location inside the source ID, but I can't find any relevant info on how is the correct way.
I would really appreciate an example of source_cosmosdb_account_id if anyone did this successfully in terraform.
Thanks
Configuration used:
backup = [
{
type = "Continuous"
interval_in_minutes = null
retention_in_hours = null
storage_redundancy = null
}
]
restore = [
{
"source_cosmosdb_account_id" = "/subscriptions/33f6-e87e-4cdf-9480-7b1dae/providers/Microsoft.DocumentDB/locations/westeu/restorableDatabaseAccounts/test-source-db-name"
"restore_timestamp_in_utc" = "2022-11-18T14:00:00.00Z"
"database" = []
}
]

I've recently faced same issue and discovered that terraform documentation is outdated.
You need to have create_mode = "Restore"
Backup should be configured like this
backup = [
{
type = "Continuous"
interval_in_minutes = 0
retention_in_hours = 0
storage_redundancy = "Geo"
}
]
Format /subscriptions/{subscriptionId}/providers/Microsoft.DocumentDB/locations/{location}/restorableDatabaseAccounts/{restorableDatabaseAccountName}
is not correct
Try with /subscriptions/{subscriptionId}/providers/Microsoft.DocumentDB/locations/{location}/restorableDatabaseAccounts/{**instanceId**}
You can find cosmos instanceId in json view

I tried to reproduce the issue in my environment.
I got the same error , Failed to parse uri /subscriptions/xxxx/providers/Microsoft.DocumentDB/locations/westeu/restorableDatabaseAccounts/test-source-db-name
with:
resource "azurerm_cosmosdb_account" "db" {
name = "tfex-cosmos-db-31960"
location = "westus2"
resource_group_name = data.azurerm_resource_group.example.name
offer_type = "Standard"
kind = "MongoDB"
create_mode = "Restore"
restore { source_cosmosdb_account_id=data.azurerm_cosmosdb_restorable_database_accounts.example.id
source_cosmosdb_account_id="/subscriptions/xxxxx/providers/Microsoft.DocumentDB/locations/westeurope/restorableDatabaseAccounts/tfex-cosmos-db-31960?api-version=2022-05-15"
restore_timestamp_in_utc="2022-11-25T22:06:00Z"
}
...
}
If I tried with westeu as the location you gave , in my case I am getting below error , as it must be westeurope as supported by azure .
Also please check
Note from cosmosdb_account | terraform registry: Database account
with Continuous type (or deleted account in last 30 days) are the
restorable accounts and there cannot be Create/Update/Delete
operations on the restorable database accounts. They can only be read and be retrieved by azurerm_cosmosdb_restorable_database_accounts.
Restore the cosmos db account is supported in azure portal , powershell,Azure CLI , ARM template as in restore-account-continuous-backup | Microsoft Learn
With azurerm provider , i can only read it through below with continuous backup type :
resource "azurerm_cosmosdb_account" "db" {
name = "tfex-cosmos-db-31960"
location = "westus2"
resource_group_name = data.azurerm_resource_group.example.name
offer_type = "Standard"
kind = "MongoDB"
enable_automatic_failover = true
capabilities {
name = "EnableAggregationPipeline"
}
capabilities {
name = "mongoEnableDocLevelTTL"
}
capabilities {
name = "MongoDBv3.4"
}
capabilities {
name = "EnableMongo"
}
consistency_policy {
consistency_level = "BoundedStaleness"
max_interval_in_seconds = 300
max_staleness_prefix = 100000
}
geo_location {
location = "eastus"
failover_priority = 0
}
backup{
type = "Continuous"
//interval_in_minutes=60
}
data "azurerm_cosmosdb_restorable_database_accounts" "example" {
name = azurerm_cosmosdb_account.db.name
location = "West Europe"
}
output "name" {
value=data.azurerm_cosmosdb_restorable_database_accounts.example.name
}
output "id" {
value = data.azurerm_cosmosdb_restorable_database_accounts.example.id
}
Output:
Try checking the same with azapi_resource block in terraform :
as it requires parameters like api version , source account id which has to come from cosmosdb_restorable_database_accounts
list and must form uri like https://management.azure.com/subscriptions/subid/providers/Microsoft.DocumentDB/locations/West US/restorableDatabaseAccounts/d9b2xxx10d?api-version=2022-05-15 which is compatible in api resource.

Related

Terraform - Add tags to failover SQL databases

I am trying to figure out a way to add tags to the failover DBs created with the help of terraform registry - azurerm_mssql_failover_group.
if I use the tags field mentioned as part of terraform documentation, it adds tags to the failover group but it does not set tags on the failover databases created.
My code for the resource group is as below
resource "azurerm_mssql_failover_group" "sql-database-failover" {
name = "sqldatabasefailover1"
server_id = "Id of Primary SQL Server"
databases = [
"Id of primary SQL dbs"
]
partner_server {
id = "Id of secondary (failover) SQL server"
}
read_write_endpoint_failover_policy {
mode = "Automatic"
grace_minutes = 60
}
tags = local.common_tags
}
I tried looking for terraform registry which can help us add tags to the already present DB but could not find any. Any help will be appreciated.
Regards Tarun
I tried to reproduce the same in my environment .
I created local tags as below and tried to add to failover group.
Code referred from : azurerm_sql_failover_group | Resources | hashicorp/azurerm | Terraform Registry
Code:
locals {
resource_tags = {
project_name = "failovergroup",
category = "devbackupresource"
}
}
resource "azurerm_mssql_server" "primary" {
name = "ka-sql-primary"
resource_group_name = data.azurerm_resource_group.example.name
location = "southeastasia"
version = "12.0"
administrator_login = "sqladmin"
administrator_login_password = "xxx"
}
resource "azurerm_mssql_server" "secondary" {
name = "ka-sql-secondary"
resource_group_name = data.azurerm_resource_group.example.name
location = "westeurope"
version = "12.0"
administrator_login = "sqladmin"
administrator_login_password = "xxx"
}
resource "azurerm_mssql_database" "db1" {
name = "kadb1"
server_id = azurerm_mssql_server.primary.id
}
resource "azurerm_mssql_failover_group" "example" {
name = "kav-example-failover-group"
server_id = azurerm_mssql_server.primary.id
databases = [azurerm_mssql_database.db1.id]
tags = local.resource_tags
partner_server {
id = azurerm_mssql_server.secondary.id
}
read_write_endpoint_failover_policy {
mode = "Automatic"
grace_minutes = 60
}
}
But the tags are not added to secondaryDb which is intended for failover group.
You can use the tags in Secondary Db resource block ” azurerm_mssql_server” for secondary as below.
locals {
resource_tags = {
...
}
}
resource "azurerm_mssql_server" "secondary" {
name = "ka-sql-secondary"
resource_group_name = data.azurerm_resource_group.example.name
location = "westeurope"
version = "12.0"
administrator_login = "sqladmin"
administrator_login_password = "pa$$w0rd"
tags = local.resource_tags
}
resource "azurerm_mssql_failover_group" "example" {
name = "kav-example-failover-group
server_id = azurerm_mssql_server.primary.id
databases = [azurerm_mssql_database.db1.id]
...
}
This created tags to my secondary sql server in azure .
Edit:
tag-support-microsoftsql
See Auto-failover groups limitations .One may need to create tags manually .
Also Note: Database restore operations don't restore the tags of the
original database.

Issue in creating sqldb in cosmosdb account on Azure with Terraform

Objective: Creating cosmosdb account and sqldb in azure with terraform
What I tried:
resource "azurerm_cosmosdb_account" "cosmosaccount" {
name = "cosmosdb"
location = var.location
resource_group_name = var.rg_name
offer_type = var.cosmosdb_offer_type
kind = var.cosmosdb_kind
is_virtual_network_filter_enabled = "true"
ip_range_filter = var.ip_range_filter
capabilities {
name = "EnableTable"
}
enable_automatic_failover = false
consistency_policy {
consistency_level = var.cosmosdb_consistancy_level
max_interval_in_seconds = 5
max_staleness_prefix = 100
}
geo_location {
location = var.location
failover_priority = 0
}
virtual_network_rule {
id = var.subnet_id
ignore_missing_vnet_service_endpoint = true
}
}
resource "azurerm_cosmosdb_sql_database" "comosdbsqldb" {
name = "driving"
resource_group_name = azurerm_cosmosdb_account.cosmosaccount.resource_group_name
account_name = azurerm_cosmosdb_account.cosmosaccount.name
throughput = 500
depends_on = [azurerm_cosmosdb_account.cosmosaccount]
}
resource "azurerm_cosmosdb_sql_container" "mdhistcontainer" {
name = "metadata_history"
resource_group_name = azurerm_cosmosdb_account.cosmosaccount.resource_group_name
account_name = azurerm_cosmosdb_account.cosmosaccount.name
database_name = azurerm_cosmosdb_sql_database.comosdbsqldb.name
partition_key_path = "/definition/id"
partition_key_version = 1
throughput = 500
indexing_policy {
indexing_mode = "consistent"
included_path {
path = "/*"
}
included_path {
path = "/included/?"
}
excluded_path {
path = "/excluded/?"
}
}
unique_key {
paths = ["/definition/idlong", "/definition/idshort"]
}
depends_on = [azurerm_cosmosdb_sql_database.comosdbsqldb]
}
Issue I am facing:
It created cosmosdb account successfully but failing at creating sqldb
Error I am getting:
Error: checking for presence of Sql Database: (Name "driving" / Database Account Name "cosmosaccount" /
Resource Group "xxxxxxxxxxxx"): documentdb.SQLResourcesClient#GetSQLDatabase:
Failure responding to request: StatusCode=405 -- Original Error: autorest/azure: Service returned an error.
Status=405 Code="MethodNotAllowed" Message="Requests for API sql are not supported for this account.
\r\nActivityId: 2d79ca83-9534-46e8-a7cf-cef1fd76e752, Microsoft.Azure.Documents.Common/2.14.0"
│
│ with module.cosmosdb.azurerm_cosmosdb_sql_database.comosdbsqldb,
│ on ../modules/cosmosdb/main.tf line 46, in resource "azurerm_cosmosdb_sql_database" "comosdbsqldb":
│ 46: resource "azurerm_cosmosdb_sql_database" "comosdbsqldb"
I have given DocumentDBContributor role also to service principal but still getting this error. I am following below documentation about syntax.. It seems I am exactly following that
https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/cosmosdb_sql_database
Please suggest how to fix this error.
I believe you are getting this error is because you are creating a Cosmos DB account targeting Table API and trying to create a database with custom name in that account.
capabilities {
name = "EnableTable"
}
With Table API, you cannot create a database with any name. There can only be a single database in the account and its name is fixed. The name of the database should be TablesDB.

Setup resource health alert for multiple storage accounts

So the idea is to create resource health alert for multiple storage accounts using terraform. It's fairly simple for one storage account as one would pass the value
output "id" {
description = "Id of the storage account created."
value = azurerm_storage_account.storage.id
}
to the resource_id or hardcode the resource id with the resource actual id. But my ask here is how I can setup one single alert block for all the storage accounts provisioned by the terraform. I have been trying to use the above data block but the name variable would take only strings. Please provide a sample example as to how you would do it.
locals {
activity_log_alerts = {
resource_health_alerts= {
environment = var.environment
resource_group_name = var.rgp
enabled = "true"
** scopes = module.main.storage_account_name["storage_name"] **
alert_name = “Resource health alert for storage accounts”
description = format(“The state of the azure resource is unknown”}
category = "ResourceHealth"
level = "Critical"
operation_name = null
resource_health = [
{
current = ["Unknown"]
previous = ["Available"]
reason = ["PlatformInitiated"]
}
]
The error I received is this: Error: Null value found in list
scopes = tolist([var.resource_id])
UPDATE: Another Approach With this approach I was hoping to get all the resource under the same RG and subscription but got the same error
data "azurerm_subscription" "current" {
subscription_id = var.subscription_id
}
locals {
activity_log_alerts = {
resource_health_alerts= {
environment = var.environment
resource_group_name = var.rgp
enabled = "true"
scopes = [data.azurerm_subscription.current.id]
alert_name = “Resource health alert for storage accounts”
description = format(“The state of the azure resource is unknown”}
category = "ResourceHealth"
level = "Critical"
operation_name = null
resource_health = [
{
current = ["Unknown"]
previous = ["Available"]
reason = ["PlatformInitiated"]
}
]
The error I received is this: Error: Null value found in list
scopes = tolist([var.resource_id])

Creating azure automation dsc configuration and dsc configuration node using terraform doesn't seems to be working

As a very first step of my release process I run the following terraform code
resource "azurerm_automation_account" "automation_account" {
for_each = data.terraform_remote_state.pod_bootstrap.outputs.ops_rg
name = "${local.automation_account_prefix}-${each.key}"
location = each.key
resource_group_name = each.value.name
sku_name = "Basic"
tags = {
environment = "development"
}
}
The automation accounts created as expected and I can see those in Azure portal.
I also have terraform code that creates a couple of windows VMs,each VM creation accompained by the following
resource "azurerm_virtual_machine_extension" "dsc" {
name = "DevOpsDSC"
virtual_machine_id = var.vm_id
publisher = "Microsoft.Powershell"
type = "DSC"
type_handler_version = "2.83"
settings = <<SETTINGS_JSON
{
"configurationArguments": {
"RegistrationUrl": "${var.dsc_server_endpoint}",
"NodeConfigurationName": "${var.dsc_config}",
"ConfigurationMode": "${var.dsc_mode}",
"ConfigurationModeFrequencyMins": 15,
"RefreshFrequencyMins": 30,
"RebootNodeIfNeeded": false,
"ActionAfterReboot": "continueConfiguration",
"AllowModuleOverwrite": true
}
}
SETTINGS_JSON
protected_settings = <<PROTECTED_SETTINGS_JSON
{
"configurationArguments": {
"RegistrationKey": {
"UserName": "PLACEHOLDER_DONOTUSE",
"Password": "${var.dsc_primary_access_key}"
}
}
}
PROTECTED_SETTINGS_JSON
}
The result is the following
So VM extension is created for each VM and the status says that provisioning succeeded.
For the next step I run the following terraform code
resource "azurerm_automation_dsc_configuration" "iswebserver" {
for_each = data.terraform_remote_state.pod_bootstrap.outputs.ops_rg
name = "iswebserver"
resource_group_name = each.value.name
automation_account_name = data.terraform_remote_state.ops.outputs.automation_account[each.key].name
location = each.key
content_embedded = "configuration iswebserver {}"
}
resource "azurerm_automation_dsc_nodeconfiguration" "iswebserver" {
for_each = data.terraform_remote_state.pod_bootstrap.outputs.ops_rg
name = "iswebserver.localhost"
resource_group_name = each.value.name
automation_account_name = data.terraform_remote_state.ops.outputs.automation_account[each.key].name
depends_on = [azurerm_automation_dsc_configuration.iswebserver]
content_embedded = file("${path.cwd}/iswebserver.mof")
}
The mof file content is the following
/*
#TargetNode='IsWebServer'
#GeneratedBy=P120bd0
#GenerationDate=02/25/2021 17:33:16
#GenerationHost=D-MJ05UA54
*/
instance of MSFT_RoleResource as $MSFT_RoleResource1ref
{
ResourceID = "[WindowsFeature]IIS";
IncludeAllSubFeature = True;
Ensure = "Present";
SourceInfo = "D:\\DSC\\testconfig.ps1::5::9::WindowsFeature";
Name = "Web-Server";
ModuleName = "PsDesiredStateConfiguration";
ModuleVersion = "1.0";
ConfigurationName = "TestConfig";
};
instance of OMI_ConfigurationDocument
{
Version="2.0.0";
MinimumCompatibleVersion = "1.0.0";
CompatibleVersionAdditionalProperties= {"Omi_BaseResource:ConfigurationName"};
Author="P120bd0";
GenerationDate="02/25/2021 17:33:16";
GenerationHost="D-MJ05UA54";
Name="TestConfig";
};
After running the code I have got the following result
The configuration is created as expected, clicking on configuration entry in UI grid, leads to the following
Meaning that node configuration is created as well. My expectation was that for each VM I will see the Node configured to run configuration provided in mof file but Nodes UI shows empty Nodes
So I was trying to configure node manually to connect all peaces together
and that fails with the following
So I am totally confisued. On the one hand there's azurerm_virtual_machine_extension that allows to create extension and bind it to the automation account. In addition there are azurerm_automation_dsc_configuration and azurerm_automation_dsc_nodeconfiguration that allows to create configuration and node configuration. But the bottom line is that you cannot connect all those dots to be able to create node.
Just to confirm that configuration is valid, I create additional vm without using azurerm_virtual_machine_extension and I was able succesfully add this MV to created node configuration
The problem was in azurerm_virtual_machine_extension dsc_configuration parameter. The value needs to be the same as name property of the azurerm_automation_dsc_nodeconfiguration resource.

How to use shared_image plan data like publisher, offer and sku in virtual_machine resource in Terraform

Using Terraform v0.12.9, azurerm provider v1.36.1
Scenario:
You have created an image based on a third-party marketplace image.
You have published this image to a shared image gallery.
You have been able to do so in an automated fashion using Terraform and Azure Devops pipelines. (ex. CIS hardened images)
For each of these previous steps you have provided plan information data. if not, you would not be able to create a VM from this image, because there is an extra cost associated with using the marketplace image.
Now you want other teams be able to use your image from the Shared Image Gallery through Terraform Automation.
How to get the plan information from the shared image without hard-coding it?
Well, I thought to just retreive the data using a data source using:
data "azurerm_shared_image" "image" {
name = var.image_name
gallery_name = var.gallery_name
resource_group_name = var.rsg_name
}
and using the necessary blocks inside the virtual_machine resource like so:
storage_image_reference {
id = data.azurerm_shared_image.image.id
}
plan {
name = data.azurerm_shared_image.image.sku
publisher = data.azurerm_shared_image.image.publisher
product = data.azurerm_shared_image.image.offer
}
However, I get an error stating:
Error: Unsupported attribute
This object has no argument, nested block, or exported attribute named "sku".
for each sku, publisher and offer.
We can get a better picture of what we are dealing with to add an output for testing purposes, like so:
output "imagedata" {
value = data.azurerm_shared_image.image
}
We get an important insight:
imagedata = {
"description" = ""
"eula" = ""
"gallery_name" = [removed]
"id" = [removed]
"identifier" = [
{
"offer" = "cis-centos-7-l1"
"publisher" = "center-for-internet-security-inc"
"sku" = "cis-centos75-l1"
},
]
"location" = [removed]
"name" = [removed]
"os_type" = "Linux"
"privacy_statement_uri" = ""
"release_note_uri" = ""
"resource_group_name" = [removed]
"tags" = {}
}
Ah now using data.azurerm_shared_image.image.identifier.sku won't just work here either. As far as Terraform is concerned, the identifier block is only one element in an array.
Solution:
We retrieve the data still the same way:
data "azurerm_shared_image" "image" {
name = var.image_name
gallery_name = var.gallery_name
resource_group_name = var.rsg_name
}
And we reference it in our virtual_machine resource like so:
storage_image_reference {
id = data.azurerm_shared_image.image.id
}
plan {
name = data.azurerm_shared_image.image.identifier[0].sku
publisher = data.azurerm_shared_image.image.identifier[0].publisher
product = data.azurerm_shared_image.image.identifier[0].offer
}

Resources