Cannot connect to DocumentDb directly after having deployed the DocumentDb account - azure

I have an ARM template that I use to deploy a DocumentDB as well as other Azure reosurces to a resource group. I want my ARM template to setup a Stream Analytics job that uses the DocumentDB as output. In order to do this the DocumentDB account created by the ARM template needs to have a database and a collection setup as well. I cannot find a way to do this from an ARM template so I have written a Powershell CmdLet to create the database and collction for me.
The Stream Analytics job cannot be created by the first ARM template since it depends on having the database and collection created first. Instead I have to divide the deployment into two ARM templates, the first setting up the DocDb account and the second setting up the SA job.
The problem is that I cannot create a database in the DocDB account directly after having deployed the account via the ARM template. I get an exception with the following message: "The remote name could not be resolved: 'test.documents.azure.com'" when I try to execute the CreateDatabaseAsync method with the DocDbEndpoint and AuthKey I get back from the ARM template deployment.
Are there any timing issues after having deployed Azure resources using a ARM template before you can access them programatically? This do not seem to be a problem with other Azure reosurces created this way.
Any help on this matter is highly appreciated as well as what is a good practice for working with ARM templates with DocumentDB and Stream Analytic jobs.
Update 2016-03-23
Code for setting up the connection to the DocumentDB to create the database.
Uri endpointUri = new Uri(documentDbEndPoint);
DocumentClient client = new DocumentClient(endpointUri, authKey);
var db = await client.CreateDatabaseAsync(new Database { Id = databaseId });
return db;
Where the documentDbEndPoint is in the form of: https://name.documents.azure.com:443/ and name is the name of my DocDB account just created by the ARM template deployment.
I have the code in a library which I can either call from a Console application or from a Powershell script by loading the library with:
Add-Type -Path <path to library dll file>
No matter if I use powershell or console application I get the same error if I try to create a database just after having created the DocDB account using the ARM template. If I wait like an hour or so both the powershell script and console application works and can create a database in the account.
Seems like there is some kind of timing issue in order for Azure to setup dns records for the newly created DocDB account so that it can be accessed using the DocDB API.
Update 2 2016-03-23
Just tried to create a DocDB account directly from the portal and doing this instead of creating it from an ARM template makes it possible to create a database in the account using my powershell script and console application immediately.

This timing issue has been fixed now and you should be able to use it from the ARM template now.

Related

Restore database on another server in azure with terraform

How do you restore an azure sql database using terraform on another server from a backup?
Terraform docs talk about a create mode "RestoreExternalBackup". How could one use that?
https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/mssql_database
I researched this issue and found it to be a discrepancy between the Terraform and Azure documentation. I also opened an issue on the GitHub repo with this info.
RestoreExternalBackup is listed as a possible value for CreateMode in the Azure API documentation for databases. However, the create mode documentation doesn't describe how to use it. This option should not be available.
Looking at the Managed Database documentation, it clearly defines how to use the RestoreExternalBackup option. Oddly enough the Terraform documentation doesnt list any create modes for managed databases. https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/mssql_managed_database
When trying to use RestoreExternalBackup to create a database the error indicates that option requires a pointer to a storage account with the message "Missing storage container URI". Storage account information is not a valid property when creating a database resource, only for a managed database resource.
Database = https://learn.microsoft.com/en-us/rest/api/sql/2022-05-01-preview/databases/create-or-update?tabs=HTTP
Managed Database Docs = https://learn.microsoft.com/en-us/rest/api/sql/2022-05-01-preview/managed-databases/create-or-update?tabs=HTTP

Upscaling/Downscaling provisioned RU for cosmos containers at specific time

As mentioned in the Microsoft documentation there is support to increase/decrease the provisioned RU of cosmos containers using cosmosDB Java SDK but when I am trying to perform the steps I am getting below error:
com.azure.cosmos.CosmosException: {"innerErrorMessage":"\"Operation 'PUT' on resource 'offers' is not allowed through Azure Cosmos DB endpoint. Please switch on such operations for your account, or perform this operation through Azure Resource Manager, Azure Portal, Azure CLI or Azure Powershell\"\r\nActivityId: 86fcecc8-5938-46b1-857f-9d57b7, Microsoft.Azure.Documents.Common/2.14.0, StatusCode: Forbidden","cosmosDiagnostics":{"userAgent":"azsdk-java-cosmos/4.28.0 MacOSX/10.16 JRE/1.8.0_301","activityId":"86fcecc8-5938-46b1-857f-9d57b74c6ffe","requestLatencyInMs":89,"requestStartTimeUTC":"2022-07-28T05:34:40.471Z","requestEndTimeUTC":"2022-07-28T05:34:40.560Z","responseStatisticsList":[],"supplementalResponseStatisticsList":[],"addressResolutionStatistics":{},"regionsContacted":[],"retryContext":{"statusAndSubStatusCodes":null,"retryCount":0,"retryLatency":0},"metadataDiagnosticsContext":{"metadataDiagnosticList":null},"serializationDiagnosticsContext":{"serializationDiagnosticsList":null},"gatewayStatistics":{"sessionToken":null,"operationType":"Replace","resourceType":"Offer","statusCode":403,"subStatusCode":0,"requestCharge":"0.0","requestTimeline":[{"eventName":"connectionAcquired","startTimeUTC":"2022-07-28T05:34:40.472Z","durationInMicroSec":1000},{"eventName":"connectionConfigured","startTimeUTC":"2022-07-28T05:34:40.473Z","durationInMicroSec":0},{"eventName":"requestSent","startTimeUTC":"2022-07-28T05:34:40.473Z","durationInMicroSec":5000},{"eventName":"transitTime","startTimeUTC":"2022-07-28T05:34:40.478Z","durationInMicroSec":60000},{"eventName":"received","startTimeUTC":"2022-07-28T05:34:40.538Z","durationInMicroSec":1000}],"partitionKeyRangeId":null},"systemInformation":{"usedMemory":"71913 KB","availableMemory":"3656471 KB","systemCpuLoad":"empty","availableProcessors":8},"clientCfgs":{"id":1,"machineId":"uuid:248bb21a-d1eb-46a5-a29e-1a2f503d1162","connectionMode":"DIRECT","numberOfClients":1,"connCfg":{"rntbd":"(cto:PT5S, nrto:PT5S, icto:PT0S, ieto:PT1H, mcpe:130, mrpc:30, cer:false)","gw":"(cps:1000, nrto:PT1M, icto:PT1M, p:false)","other":"(ed: true, cs: false)"},"consistencyCfg":"(consistency: Session, mm: true, prgns: [])"}}}
at com.azure.cosmos.BridgeInternal.createCosmosException(BridgeInternal.java:486)
at com.azure.cosmos.implementation.RxGatewayStoreModel.validateOrThrow(RxGatewayStoreModel.java:440)
at com.azure.cosmos.implementation.RxGatewayStoreModel.lambda$toDocumentServiceResponse$0(RxGatewayStoreModel.java:347)
at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:106)
at reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onNext(FluxSwitchIfEmpty.java:74)
at reactor.core.publisher.FluxPeek$PeekSubscriber.onNext(FluxPeek.java:200)
at reactor.core.publisher.FluxHandle$HandleSubscriber.onNext(FluxHandle.java:119)
Message says to switch on such operations for your accounts but I could not find any page to do that. Can I use Azure functions to do the same thing at a specific time?
Code snippet:
CosmosAsyncContainer container = client.getDatabase("DatabaseName").getContainer("ContainerName");
ThroughputProperties autoscaleContainerThroughput = container.readThroughput().block().getProperties();
container.replaceThroughput(ThroughputProperties.createAutoscaledThroughput(newAutoscaleMaxThroughput)).block();
This is because disableKeyBasedMetadataWriteAccess is set to true on the account. You will need to contact either your subscription owner or someone with DocumentDB Account Contributor to modify the throughput using PowerShell or azure cli, links to samples. You can also do this by redeploying the ARM template or Bicep file used to create the account (be sure to do a GET first on the resource so you don't accidentally change something.
If you are looking for a way to automatically scale resources up and down on a schedule, please refer to this sample here, Scale Azure Cosmos DB throughput by using Azure Functions Timer trigger
To learn more about the disableKeyBasedMetadataWriteAccess property and it's impact to control plane operations from the data plane SDK's see, Preventing changes from the Azure Cosmos DB SDKs

Updating Set of Values in ARM Templates Automatically

I have an ARM template, which I will be using it to deploy resources (Mentioned as in Azure Portal) via Azure DevOps Pipeline. These ARM templates were created using my Dev Subscription. If I need to use this same ARM template to move to production, I need to manually update the subscription id etc in the ARM template and then run the respective pipelines. Is there any way to automate this manual updation process in ARM template.
I have tried using File Transform Agent job available under Azure DevOps Release pipeline. But since the values are available inside nested loops of ARM templates, it failed. Is there any PowerShell script that will be suitable for this process of updating values.
I don't want to manually update the ids under the ARM Template. Instead, I want to update it automatically.
Set Json Property task might help to set the property with specific value.
First in the marketplace search for "set Json property" task and install it for your organization.
Then you can set the property path and the value accordingly. check here for detail usage
Instead of hard coding in ARM templates you can use parameters file. For dev and prod you can have separate files and while deploying pass ARM Template along with parameter file whichever is required.
You can use AKV (Azure Key Vault) to access your subscription details in parameter file
subscription().subscriptionid will get you the details

Azure Template reference function does not work for existing resource (not in template)

trying to use reference function in my ARM template for a resource that is already existing, but not included in my template. This does not seem to work. Is there some way to do this?
In particular, I'm deploy a new SQL database to an existing SQL server, along with a new webapp. In the web app, I'm trying to set connection string using
reference(concat('Microsoft.Sql/servers/', variables('sqlserverName'))).fullyQualifiedDomainName
but this doesnot work - says the resource is not available/included.
For reference function to work on existing resources you need to pass in the API versiĆ³n. Documentation.
apiVersion - API version of the specified resource. Include this parameter when the resource is not provisioned within same template. Typically, in the format, yyyy-mm-dd.
reference(parameters('storageAccountName'), '2016-12-01')

ARM Template listkeys() fails to find Azure Search

I'm creating an ARM Template to deploy both an Azure Search instance and a Function App that depends on the Azure Search instance. As part of that I'm trying to pre-populate the Function Apps app settings with the Search Service's API Key.
I'm doing this by adding the following into the value of one of the app settings of the FunctionAppSite:
[listKeys(variables('searchServiceId'), '2015-08-19').key1]
I get the searchServiceId in the variables sections
[resourceId('Microsoft.Search/searchServices', parameters('SearchServiceName'))]
The FunctionAppSite component lists dependencies on the AppServicePlan, StorageAccount and SearchService.
When I deploy the template the Search Service gets created then an error occurs because listkeys couldn't find the Search Service. This happens about .25s after the service is created.
I'm pretty sure I have the syntax around the listkeys correct and the problem is just the timing, no sure how I can slow it down though. I tried forcing the StorageAccount to depend on the SearchService in my template, hoping the dependency chain would slow things down enough, but the listkeys error happens after the Search Service is created, but before the StorageAccount is created.
The full template can be found here and there is a 'Deploy to Azure' button on the readme.md of that repo if you want to see it in action.
Well, looking at provider operations for Microsoft.Search:
Microsoft.Search/register/action
Microsoft.Search/checkNameAvailability/action
Microsoft.Search/searchServices/write
Microsoft.Search/searchServices/read
Microsoft.Search/searchServices/delete
Microsoft.Search/searchServices/start/action
Microsoft.Search/searchServices/stop/action
Microsoft.Search/searchServices/listAdminKeys/action
Microsoft.Search/searchServices/regenerateAdminKey/action
Microsoft.Search/searchServices/createQueryKey/action
Microsoft.Search/searchServices/queryKey/read
Microsoft.Search/searchServices/queryKey/delete
this is how you do it:
"[listAdminKeys(variables('searchServiceId'), '2015-08-19').PrimaryKey]"
To list provider operations:
$ops = (Get-AzureRmProviderOperation -OperationSearchString */*).Operation

Resources