Get Cosmosdb Container Collection items using powershell - azure

Team, I had created new CosmosDB account in Azure portal with a container contains list of collection items. I am able to access Container details in power shell script.
How to list collection items or show specific collection item using partition key using power shell script
Power shell Script :
Get-AzResource -ResourceType "Microsoft.DocumentDB/databaseAccounts/sqlDatabases/containers" -ApiVersion "2020-03-01" -ResourceGroupName "testRG" -Name "cosmosaccount1/database1/containercollection1"

You would need to use something like a thirdparty module for this. Azure Resource Manager doesnt support that, hence you need to talk to Cosmos DB directly.
https://github.com/PlagueHO/CosmosDB

The Cosmos DB repo has a set of examples to use Powershell: https://github.com/Azure/azure-cosmos-dotnet-v3/tree/master/Microsoft.Azure.Cosmos.Samples/Usage/PowerShellRestApi
Particularly to read Items: https://github.com/Azure/azure-cosmos-dotnet-v3/blob/master/Microsoft.Azure.Cosmos.Samples/Usage/PowerShellRestApi/PowerShellScripts/ReadItem.ps1
They are all using the REST API to do REST request, in this case, it is an authenticated GET to https://{databaseaccount}.documents.azure.com/dbs/{db-id}/colls/{coll-id}/docs/{id} (where databaseaccount is your account name, db-id is the id of your database, coll-id is the id of your collection/container, and id is your document id). It is also setting the x-ms-documentdb-partitionkey header for the partition key.

Like #4c74356b41 has indicated, you can use the CosmosDB module, which is now part of the official Az module.
Install-Module -Name Az -Scope CurrentUser -Repository PSGallery -Force
You can see the available commands with Get-Commands:
Import-Module Az
Import-Module -Name CosmosDB
Get-Command -Module CosmosDB
Get all items in a collection
In order to get all entries inside a container, we use the Get-CosmosDbDocument command:
$subscription = "SubscriptionName"
$resourceGroupName = "ResourceGroupName"
$accountName = "AzureCosmosDBAccount"
$databaseName = "DatabaseName"
$cosmosContainer = "TargetCosmosDBContainer"
Set-AzContext $subscription
$backOffPolicy = New-CosmosDbBackoffPolicy -MaxRetries 5 -Method Additive -Delay 1000
$cosmosDbContext = New-CosmosDbContext -Account $accountName -Database
$databaseName -ResourceGroup $resourceGroupName -BackoffPolicy $backOffPolicy
$documentsPerRequest = 100
$continuationToken = $null
$documents = $null
do {
$responseHeader = $null
$getCosmosDbDocumentParameters = #{
Context = $cosmosDbContext
CollectionId = $cosmosContainer
MaxItemCount = $documentsPerRequest
ResponseHeader = ([ref] $responseHeader)
}
if ($continuationToken) {
$getCosmosDbDocumentParameters.ContinuationToken = $continuationToken
}
$documents += Get-CosmosDbDocument #getCosmosDbDocumentParameters
$continuationToken = Get-CosmosDbContinuationToken -ResponseHeader $responseHeader
} while (-not [System.String]::IsNullOrEmpty($continuationToken))
Note: There is no apparent limitation on the number of documents that can be retrieved with this command, but it stands to reason that the command will have the API limitation, and this is 4 MB (as documented here). The value here ($documentsPerRequest = 100) could prove to be either too big or too small, depending on the size of each document. I usually don't use this parameter, but I've mentioned it here in case someone needs it.
List specific collection item
To get a specific entry or group of entries from a container, we use the same Get-CosmosDbDocument command, in a slightly different way:
$query = "SELECT * FROM c WHERE c.property = 'propertyValue'"
$documents = Get-CosmosDbDocument -Context $cosmosDbContext -CollectionId $cosmosContainer -Query $query -QueryEnableCrossPartition $true
Note: For brevity I haven't went to the process of getting a continuation token, but if the query will return a result that is larger than 4 MB, then we will only receive the first part of the response. To make sure this does not happen we should add "Query" and "QueryEnableCrossPartition" in the $getCosmosDbDocumentParameters dictionary.

Related

Azure monitor alert multiple subscriptions

I need to create multiple azure monitor alarms.
I'm trying to do this following this website:
https://www.azureblue.io/how-to-create-an-alert-rule-using-powershell-and-azure-cli/
This solution works for me if I have action group and target resources in the same subscription but generally I don't.
I have one actiongroup and hundreds targets in almost 100 subscriptions.
I have:
#jump to subscription with actiongroup
$context = Get-AzSubscription -SubscriptionId xxx
Set-AzContext $context
$actionGroup = Get-AzActionGroup -name "xxx" -ResourceGroupName "xxx"
$actionGroupId = New-AzActionGroup -ActionGroupId $actionGroup.Id
#jump to subscription with target without this I can't ask for target.id
$context = Get-AzSubscription -SubscriptionId xx
Set-AzContext $context
# Creates a local criteria object that can be used to create a new metric alert
$condition = New-AzMetricAlertRuleV2Criteria `
-MetricName "Data space used percent (Platform)" `
-TimeAggregation Maximum `
-Operator GreaterThan `
-Threshold 0.8
$windowSize = New-TimeSpan -Minutes 30
$frequency = New-TimeSpan -Minutes 5
$targetResourceId = (Get-AzResource -Name xxx).Id
#I was thinking that this jump to subscription will solve my issue but doesn't
$context = Get-AzSubscription -SubscriptionId cf653672-c304-49a2-b01b-171f6236bad6
Set-AzContext $context
# Adds or updates a V2 (non-classic) metric-based alert rule.
Add-AzMetricAlertRuleV2 `
-Name "test" `
-ResourceGroupName "xxx" `
-WindowSize $windowSize `
-Frequency $frequency `
-TargetResourceId $targetResourceId `
-Condition $condition `
-ActionGroup $actionGroupId `
-Severity 3
I got this error:
Add-AzMetricAlertRuleV2 : Exception type: ErrorResponseException, Message: Null/Empty, Code: Null, Status code:NotFound, Reason phrase: Not Found
At line:26 char:1
+ Add-AzMetricAlertRuleV2 `
+ ~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : CloseError: (:) [Add-AzMetricAlertRuleV2], PSInvalidOperationException
+ FullyQualifiedErrorId : Microsoft.Azure.Commands.Insights.Alerts.AddAzureRmMetricAlertRuleV2Command
From GUI there is no a problem to chose actiongroup from different subscription but you can't chose targets from multiple subscription.
If I will find solution for this one example I will try to read all my resources and run the code in a loop.
I had an error like that recently.. It was due to an invalid input. In my particular case it was due to invalid characters in the alertName.
As for the main issue of multiple subscriptions...
Here are a couple of ideas:
To apply an single metric alert to multiple resources of the same type as the same time. You can specify scope, region etc with Add-AzMetricAlertRuleV2 see:
https://learn.microsoft.com/en-us/powershell/module/az.monitor/add-azmetricalertrulev2?view=azps-7.2.0. This is limited to only a few resources https://learn.microsoft.com/en-us/azure/azure-monitor/alerts/alerts-metric-overview#monitoring-at-scale-using-metric-alerts-in-azure-monitor.
Note that TargetResourceScope is a array of strings. So it seems possible in theory to apply the alert to multiple subscriptions with this.
When other option are not possible: use Get-AzSubscription, and deploy the code to each subscription or to get the list of subscriptions you need to run the code with.
Your single action group.
I can validate that, you can have a single action group for any alerts inside a subscription. I don't see any reason why the the action group could not be in a separate subscription provided the actiongroup.id is correct and there are no RBAC/firewall issues. But I don't have a way to test this.
Lastly, here is a relevant conversation in an MS forum. https://techcommunity.microsoft.com/t5/azure-monitor/azure-monitor-multiple-subscriptions/m-p/1348362

Azure Powershell Positional Parameter error

I'm trying to do a deployment using powershell. Both parameter and template file are stored in blob storage, but I get the error below before it even tries to download the blobs.
New-AzResourceGroupDeployment : A positional parameter cannot be found that accepts argument 'newdeployment-878059'. At C:\Temp\New-Deployment\deploy-core.ps1:86 char:1
The code I use is below
$vnetRG = "rg-vnet"
$vpnRG = "rg-vpn"
$fwRG = "rg-fw"
$btnRG = "rg-bastion"
$loc = "west europe"
New-AzResourceGroupDeployment -Name = "newdeployment-$random"-ResourceGroupName "rg-vnet" `
-TemplateParameterFile "$PSScriptRoot\core.parameters.json" -TemplateUri = $templateFileUri `
-vpnResourceGroupName $rgRG -vpnResourceGroupName $vpnRG -fwResourceGroupName $fwRG -btnResourceGroupName $btnRG
I'm trying to deploy multiple resources to various resource groups in one subscription.
Thanks in advance :)
Here my comment as answer
In your code, you have added an = character between some of the parameter names and the actual value to use for that parameter.
In PowerShell you assign a value to a parameter with a space character between the two.
Also, there are parameters used that (according to the docs) don't exist for that cmdlet, like vpnResourceGroupName, fwResourceGroupName and btnResourceGroupName. To me, they sound like variables mistakenly added as parameter with the leading $ stripped off?
For cmdlets that can potentially use a large number of parameters, I'd recommend using Splatting, to keep the code clean and easy to maintain.
Something like:
$splatParams = #{
Name = "newdeployment-$random"
ResourceGroupName = "rg-vnet"
TemplateParameterFile = "$PSScriptRoot\core.parameters.json"
TemplateUri = $templateFileUri
}
New-AzResourceGroupDeployment #splatParams

azure powerhsell integration account partner creation identities invalid

I have a paid and free integration account setup on Azure. I have already generated a few partners and agreements using the portal with no issues. Im now looking to use some powershell scripts to help control the creation of these partners and agreements in the future.
When trying to build script to create a partner I hit this bottleneck with the business identities.
New-AzIntegrationAccountPartner -ResourceGroupName $ResourceGroupName …
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| Invalid business identity.
$ResourceGroupName = 'rg-test'
$IntegrationAccountName = 'inter-test'
$PartnerName = 'TestPartner'
$BusinessIdentities = #{
Qualifier = "AS2Identity"
Value = "TestIdentity"
}
New-AzIntegrationAccountPartner -ResourceGroupName $ResourceGroupName -Name $IntegrationAccountName -PartnerName $PartnerName -BusinessIdentities $BusinessIdentities
Reference: https://learn.microsoft.com/en-us/powershell/module/az.logicapp/new-azintegrationaccountpartner?view=azps-4.6.1
Is there something Im unaware when it comes to hash-tables?
Powershell Core MacOS 10.15.6
Major Minor Patch PreReleaseLabel BuildLabel
7 0 3
Update:
Adding Resolve-AzError Response.
HistoryId: 4
Message : Invalid business identity.
StackTrace : at Microsoft.Azure.Commands.LogicApp.Utilities.CmdletHelper.ConvertToBusinessIdentityList(Object businessIdentityObject)
at Microsoft.Azure.Commands.LogicApp.Cmdlets.NewAzureIntegrationAccountPartnerCommand.ExecuteCmdlet()
at Microsoft.WindowsAzure.Commands.Utilities.Common.CmdletExtensions.<>c__3`1.<ExecuteSynchronouslyOrAsJob>b__3_0(T c)
at Microsoft.WindowsAzure.Commands.Utilities.Common.CmdletExtensions.ExecuteSynchronouslyOrAsJob[T](T cmdlet, Action`1 executor)
at Microsoft.WindowsAzure.Commands.Utilities.Common.CmdletExtensions.ExecuteSynchronouslyOrAsJob[T](T cmdlet)
at Microsoft.WindowsAzure.Commands.Utilities.Common.AzurePSCmdlet.ProcessRecord()
Exception : System.Management.Automation.PSArgumentException
InvocationInfo : {New-AzIntegrationAccountPartner}
Line : New-AzIntegrationAccountPartner -ResourceGroupName $ResourceGroupName -Name $IntegrationAccountName -PartnerName $PartnerName -BusinessIdentities
$BusinessIdentities
Position : At /Users/john/Desktop/run.ps1:10 char:1
+ New-AzIntegrationAccountPartner -ResourceGroupName $ResourceGroupName …
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
HistoryId : 4
According to my research, when we run the command New-AzIntegrationAccountPartner, we need to define BusinessIdentities as Array. Becase the code Microsoft.Azure.Commands.LogicApp.Utilities.CmdletHelper.ConvertToBusinessIdentityList(Object businessIdentityObject) in the command New-AzIntegrationAccountPartner need users to provide Array object. Otherwise, it will throw error. For more details, please refer to here and here.
For example
$ResourceGroupName = 'testaks'
$IntegrationAccountName = 'test06'
$PartnerName = 'TestPartner'
$BusinessIdentities = #("<the Qualifier's value such as AS2Identity>","<The Value's value>")
New-AzIntegrationAccountPartner -ResourceGroupName $ResourceGroupName -Name $IntegrationAccountName -PartnerName $PartnerName -BusinessIdentities $BusinessIdentities

How to find the number of objects in an Azure blob container - Archive Tier

We are trying to re-hydrate some compressed backup files from Azure Archive tier to Cool tier. We are not concerned about the price of the data size itself (10TB). What we are concerned about is the potential I/O charges of moving tiers of individual files. The back-up process saves each job in "chunks" (i.e. folders) which contain a file tree of individual compressed blocks, varying in size. There could be a 100 thousand or a 100 billion of these individual blocks in the entire 10TB, we don't know.
Are there any PowerShell scripts/commands that could give the count of individual objects in the entire 10TB blob container? If we could determine the count of objects we could then calculate the I/O costs. We have command-line access and know that some of the API calls only return results in 5000 item blocks. We also know that any recursive search could take a lot of time.
$storageAccount = Get-AzStorageAccount `
-ResourceGroupName $resourceGroup `
-Name $storageAccountName
$blobCount = 0
$token = $null
$MaxReturn = 5000
do {
$Blobs = Get-AzStorageBlob -Context $storageAccount.Context -Container $containerName `
-MaxCount $MaxReturn -ContinuationToken $token
if ( -not $Blobs ) { break } # not sure if this is needed
$blobCount = $blobCount + $Blobs.Count
$token = $Blobs[$Blobs.Count - 1].ContinuationToken;
} while ( $token )
this is the "script" I came up with. based on this

Powershell getting same values when using if in foreach

I'm trying to get metric from Azure to Zabbix.
The issue is that Metric for VM consists of 2 words:Percentage CPU, and Zabbix doesn't allow item keys to consists of 2 words. I also tried Percentage%20CPU but getting errors in Zabbix, and I created Zabbix key percentage_cpu.
So I decided prior sending data from Zabbix to Azure to "translate" percentage_cpu to Percentage%20CPU. This works great if only that key is present, but issue starts when I add another key (in this example SQL metric).
For SQL metric all values are in one word - no need to change anything, but then metric for VM is also assigned to SQL. I'm trying to avoid writing separate file for every service
$host_items = Get-ZabbixHostItems -url $zabbix_url -auth $zabbix_auth -
zabbix_host $host_name
foreach ($host_item in $host_items)
{
#$host_item_details = select-string -InputObject $host_item.key_ -Pattern '^(azure\.sql)\.(.*)\.(.*)\[\"(.*)\"\]$';
$host_item_details = select-string -InputObject $host_item.key_ -Pattern '^(azure\.\w{2,})\.(.*)\.(.*)\[\"(.*)\"\,(.*)]$';
#$host_item_details = select-string -InputObject $host_item.key_ -Pattern '^(azure)\.(.*)\.(.*)\.(.*)\[\"(.*)\"\,(.*)]$';
$host_item_provider = $host_item_details.Matches.groups[1];
$host_item_metric = $host_item_details.Matches.groups[2];
$host_item_timegrain = $host_item_details.Matches.groups[3];
$host_item_resource = $host_item_details.Matches.groups[4];
$host_item_suffix = $host_item_details.Matches.groups[5];
if ($host_item_metric='percentage_cpu')
{$host_item_metric='Percentage%20CPU'}
else
{ $host_item_metric = $host_item_details.Matches.groups[2];}
#}
$uri = "https://management.azure.com{0}/providers/microsoft.insights/metrics?api-version={1}&interval={2}&timespan={3}&metric={4}" -f `
$host_item_resource, `
"2017-05-01-preview", `
$host_item_timegrain.ToString().ToUpper(), `
$($(get-date).ToUniversalTime().addminutes(-15).tostring("yyyy-MM-ddTHH:mm:ssZ") + "/" + $(get-date).ToUniversalTime().addminutes(-2).tostring("yyyy-MM-ddTHH:mm:ssZ")), `
$host_item_metric;
write-host $uri;
}
output of hostitems_
azure.sql.dtu_consumption_percent.pt1m["/subscriptions/111-222/resourceGroups/rg/providers/Microsoft.Sql/servers/mojsql/databases/test",common]
azure.vm.percentage_cpu.pt1m["/subscriptions/111-222/resourceGroups/rg/providers/Microsoft.Compute/virtualMachines/test",common]
When I ran code above I'm getting these URI's
https://management.azure.com/subscriptions/111-222/resourceGroups/rg/providers/Microsoft.Sql/servers/mojsql/databases/test/providers/microsoft.insights/m
etrics?api-version=2017-05-01-preview&interval=PT1M&timespan=2018-08-11T07:38:05Z/2018-08-11T07:51:05Z&metric=Percentage%20CPU
https://management.azure.com/subscriptions/111-222/resourceGroups/rg/providers/Microsoft.Compute/virtualMachines/test/providers/microsoft.insights/metric
s?api-version=2017-05-01-preview&interval=PT1M&timespan=2018-08-11T07:38:05Z/2018-08-11T07:51:05Z&metric=Percentage%20CPU
For first link (SQL) metric should be dtu_consumption but I'm getting same metric for both links
Second attempt:
if ($host_item_metric -eq 'percentage_cpu')
{$host_item_metric='Percentage%20CPU';}
else
{ $host_item_metric = $host_item_details.Matches.groups[2];}
write-host $host_item_metric
}
output: (original values)
dtu_consumption_percent
percentage_cpu
had to use -like
if ($host_item_metric -like 'percentage_cpu')
{$host_item_metric='Percentage%20CPU';}
else
{ $host_item_metric = $host_item_details.Matches.groups[2]}

Resources