During the cleanup process of Azure VM. I am trying to identify all resources associated with Deallocated VM, Like network, storage and nic.
I ran below query to get details, but unable to write a query to get other details in the same query parameter to get a result of nic, storge in table format.
az vm list -d --query "[?powerState=='VM deallocated']" -o table
qa-automation-10 TEST-QA-AUTOMATION
qa-automation-11 TEST-QA-AUTOMATION
qa-automation-12 TEST-QA-AUTOMATION
qa-automation-13 TEST-QA-AUTOMATION
qa-automation-14 TEST-QA-AUTOMATIO
Any help will be appreciable, I am especially looking for az client query. As VM deallocated list is big so I will run through gitlab pipeline.
},
"id": "/subscriptions/xxxxxxxx/resourceGroups/xxxx/providers/Microsoft.Compute/virtualMachines/x023901",
"identity": null,
"licenseType": null,
"location": "x",
"macAddresses": "",
"name": "x023901",
"networkProfile": {
"networkInterfaces": [
{
"id": "/subscriptions/x/resourceGroups/rGroup-ENV0239/providers/Microsoft.Network/networkInterfaces/x023901nic",
"primary": null,
"resourceGroup": "rGroup-ENV0239"
}
]
},
"osProfile": null,
"plan": null,
"powerState": "VM deallocated",
"privateIps": "x.x.x.x",
"provisioningState": "Succeeded",
"publicIps": "",
"resourceGroup": "RGROUP-ENV0239",
"resources": [
{
"autoUpgradeMinorVersion": true,
"forceUpdateTag": null,
"id": "/subscriptions/xxxxxxx/resourceGroups/RGROUP-ENV0239/providers/Microsoft.Compute/virtualMachines/x023901/extensions/OmsAgentForLinux",
"instanceView": null,
"location": "x",
"name": "OmsAgentForLinux",
"protectedSettings": null,
"provisioningState": "Succeeded",
"publisher": "Microsoft.EnterpriseCloud.Monitoring",
"resourceGroup": "RGROUP-ENV0239",
"settings": {
"stopOnMultipleConnections": true,
"workspaceId": "xx"
},
"tags": null,
"type": "Microsoft.Compute/virtualMachines/extensions",
"typeHandlerVersion": "1.0",
"virtualMachineExtensionType": "OmsAgentForLinux"
}
],
"storageProfile": {
"dataDisks": [
{
"caching": "None",
"createOption": "Attach",
"diskSizeGb": 20,
"image": null,
"lun": 0,
"managedDisk": null,
"name": "x-data1.vhd",
"vhd": {
"uri": "https://x.core.windows.net/vhds/x-data1.vhd"
},
"writeAcceleratorEnabled": null
}
],
"imageReference": null,
"osDisk": {
"caching": "ReadWrite",
"createOption": "Attach",
"diffDiskSettings": null,
"diskSizeGb": 30,
"encryptionSettings": null,
"image": null,
"managedDisk": null,
"name": "xosDisk",
"osType": "Linux",
"vhd": {
"uri": "https://xblob.core.windows.net/vhds/x.vhd"
},
"writeAcceleratorEnabled": null
}
},
"tags": null,
"type": "Microsoft.Compute/virtualMachines",
"vmId": "x",
"zones": null
},
The VMs in Azure has two types, one is the managed VM while the other is unmanaged VM. When you want to show the details of all the VMs, you should pay attention to this.
In addition, there no property about the storage account. You can store the files in the storage account and do not associate to the VM. So you just get a little info about the storage account if the VM is unmanaged.
Get a list of the VM with some resources just like vmName, NIC, osDisk or osDiskURI, the CLI command here:
az vm list --query "[].{VMName:name, nicId:networkProfile.networkInterfaces[0].id, managedDiskId:storageProfile.osDisk.managedDisk.id, UnmanagedDiskURL:storageProfile.osDisk.vhd.uri}" -o table
You can change the info that you want if you can find it in the VM details through the CLI command az vm show. Hope this will help. Any more question you can give me the message.
You will need to use a query like below:
az vm show -g RG-DemoAutomation -n DemoVM101 --query "{VMName:name, admin:osProfile.adminUsername,nicId:networkProfile.networkInterfaces[0].id, osDiskId:storageProfile.osDisk.managedDisk.id}" -o table
As you mentioned you will need to iterate through the output of the az vm list and then for each VM you will need to run az vm show as I showed in the sample above.
You can have filter and data selection in one query as "[?query here].{customColName1:value1, customColName2:value2}" but the problem is that az vm list does not have the details that you are looking for. That detail is available with az vm show command.
For values in the above syntax, the query uses JMESPATH and you can find the exact value by using a neat utility available here: JMESPath Terminal
Also, try using the output as json (-o json) as that will be much more readable unless you want to dump the output into an excel sheet.
Related
I am using copy to create multiple VM's and want to have multiple datadisks for each vm.
I know i can do it this way:
{
"name": "[concat('dataDisk-',parameters('vm-name'),'-0',copyIndex(1))]",
"diskSizeGB": "[parameters('dataDisksize')]",
"lun": 0,
"createOption": "Empty"
},
{
"name": "[concat('dataDisk1-',parameters('vm-name'),'-0',copyIndex(1))]",
"diskSizeGB": "[parameters('dataDisksize')]",
"lun": 1,
"createOption": "Empty"
}
Now I have to create 20 disks with the same name and this doesn't seem to be good workaround. Getting issue with disk name while using copy. For my case i can use the same disksize for all 20 disks
You can use the copy element to repeat properties:
"copy": [
{
"name": "dataDisks",
"count": "[parameters('numberOfDataDisks')]",
"input": {
"lun": "[copyIndex('dataDisks')]",
"createOption": "Empty",
"diskSizeGB": 1023
}
}
]
More information here: https://learn.microsoft.com/en-us/azure/azure-resource-manager/templates/copy-properties
Now I have to create 20 disks with the same name and this doesn't seem
to be good workaround. Getting issue with disk name while using copy.
Creating multiple disks using same name is impossible. you need concat either copyindex values to disk name or the VMnames to the Names of the disk..
In our template we have used the copy function block in the below manner to deploy multiple disks for a single virtual Machine & we have tested this in our local environment which is working fine.
Here is the ARM template that we have created :
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"dataDiskCount": {
"type": "int","minvalue" : 1,"maxValue" :20,"defaultValue" : 4,
"metadata": {
"description": "number of data disks."
}
}
},
"variables": {}
"resources": [
{
"copy": [
{
"name": "dataDisks",
"count": "[parameters('dataDiskCount')]",
"input": {
"diskSizeGB": "20",
"lun": "[copyIndex('dataDisks')]",
"createOption": "Empty",
"name":"[concat('disk',copyIndex('dataDisks'),'dk')]"
}
}
},
"outputs": {}
}
Here is the sample output screen shot for reference:
I'm deploying Azure function into Premium Function Plan (Elastic) using Azure powershell script:
New-AzResourceGroupDeployment -ResourceGroupName $RESOURCE_GROUP -TemplateFile "function-app.json" -TemplateParameterObject $params -Name $APP_SERVICE_NAME -Mode Incremental > $null
And deployment ignores my preWarmedInstanceCount setting. Newly created function has Always Ready Instances = 0 (see screenshot)
ARM template of function:
{
"apiVersion": "2020-06-01",
"name": "[parameters('siteName')]",
"type": "Microsoft.Web/sites",
"identity": {
"type": "systemAssigned"
},
"kind": "functionapp",
"location": "[resourceGroup().location]",
"properties": {
"name": "[parameters('siteName')]",
"serverFarmId": "[resourceId(parameters('appServicePlanRg'),'Microsoft.Web/serverfarms',parameters('appServicePlanName'))]",
"clientAffinityEnabled": false,
"siteConfig": {
"use32BitWorkerProcess": false,
"preWarmedInstanceCount": 2,
"appSettings": [
{
"name": "FUNCTIONS_EXTENSION_VERSION",
"value": "~3"
},
{
"name": "WEBSITE_CONTENTAZUREFILECONNECTIONSTRING",
"value": "DefaultEndpointsProtocol=https;AccountName=xxx..."
},
{
"name": "WEBSITE_CONTENTSHARE",
"value": "asggas"
}
]
}
}
}
It seems to me few days ago it did worked properly and I managed to set that value via arm template and now I can only update it via Azure portal.
Here is ARM template of my hosting plan:
{
"type": "Microsoft.Web/serverfarms",
"apiVersion": "2018-02-01",
"name": "[parameters('appServicePlanName')]",
"location": "[resourceGroup().location]",
"properties": {
"name": "[parameters('appServicePlanName')]",
"workerSize": "1",
"numberOfWorkers": "1",
"maximumElasticWorkerCount": 20
},
"sku": {
"Tier": "ElasticPremium",
"Name": "EP2"
}
}
After some investigation I've come to "Activity Log" (after manual update on portal) screen and was surprised that my desired property is named as "minimumElasticInstanceCount" (it is not documented anywhere in either ARM API version https://learn.microsoft.com/en-us/azure/templates/microsoft.web/sites)
Then I added this field to my ARM template and all looks good for now. Also some explanation and difference between "Always Ready Instances"(minimumElasticInstanceCount) and "Pre-warmed instances"(preWarmedInstanceCount) are posted here: https://learn.microsoft.com/uk-ua/azure/azure-functions/functions-premium-plan
So in Azure Portal Pre-warmed instance setting is not displayed. And I was looking for another setting.
I have tried to replicate the issue and was able to figure out the issue here.
Instead of "preWarmedInstanceCount" try "reservedInstanceCount".
You can check from the Azure portal itself. Once you have updated the reserved instance count from the portal you can export the template :
Then compare your previous template with the exported one, you will be able to see the difference.
I am currently deploying a VM Scale Set (VMSS) using an ARM template which has a resource inside VMSS to install Azure extension for Azure DevOps (ADO) Deployment Agent. All is deployed successfully and a node is registered in ADO with all details as are in the ARM template. However the problem is that it installs the agent only on first node and (as far as I see) ignores the rest of the nodes. I've tested this with multiple nodes during creation of the scale set and with auto-scale as well. Both scenarios result in only first agent registered.
This is the code layout I'm using (I've removed the VMSS bits to reduce the template length here, there are of course OS, storage and network settings inside):
{
"type": "Microsoft.Compute/virtualMachineScaleSets",
"name": "[parameters('VMSSName')]",
"apiVersion": "2018-10-01",
"location": "[resourceGroup().location]",
"sku": {
"name": "[parameters('VMSSSize')]",
"capacity": "[parameters('VMSSCount')]",
"tier": "Standard"
},
"dependsOn": [],
"properties": {
"overprovision": "[variables('overProvision')]",
"upgradePolicy": {
"mode": "Automatic"
},
"virtualMachineProfile": {},
"storageProfile": {},
"networkProfile": {},
"extensionProfile": {
"extensions": [
{
"type": "Microsoft.Compute/virtualMachineScaleSets/extensions",
"name": "VMSS-NetworkWatcher",
"location": "[resourceGroup().location]",
"properties": {
"publisher": "Microsoft.Azure.NetworkWatcher",
"type": "[if(equals(parameters('Platform'), 'Windows'), 'NetworkWatcherAgentWindows', 'NetworkWatcherAgentLinux')]",
"typeHandlerVersion": "1.4",
"autoUpgradeMinorVersion": true
}
},
{
"type": "Microsoft.Compute/virtualMachineScaleSets/extensions",
"name": "VMSS-TeamServicesAgent",
"location": "[resourceGroup().location]",
"properties": {
"publisher": "Microsoft.VisualStudio.Services",
"type": "[if(equals(parameters('Platform'), 'Windows'), 'TeamServicesAgent', 'TeamServicesAgentLinux')]",
"typeHandlerVersion": "1.0",
"autoUpgradeMinorVersion": true,
"settings": {
"VSTSAccountName": "[parameters('VSTSAccountName')]",
"TeamProject": "[parameters('VSTSTeamProjectName')]",
"DeploymentGroup": "[parameters('VSTSDeploymentGroupName')]",
"AgentName": "[concat(parameters('VMSSName'),'-DG')]",
"Tags": "[parameters('VSTSDeploymentAgentTags')]"
},
"protectedSettings": {
"PATToken": "[parameters('VSTSPATToken')]"
}
}
}
]
}
}
}
}
Now the desired state, of course, is that all nodes will have agent installed so that I can use the Deployment Group inside Release pipeline.
your problem is in the fact that all agents have the same AgentName, so it effectively overwrites the agent and only the latest one "survives". I dont think there is anything you can do, unless you just amend the AgentName and it auto assigns based on computer name.
You can convert this to a script\dsc extension, that way you can calculate everything on the fly.
The question in the title: "how to create VM with Json arm template using define VHD (or disk,snapshot) in azure?" Thanks for your help!
You can create the vm from VHD file through the template. The steps are that you can upload the VHD file to an Azure storage account and create a managed disk from the VHD file. Then attach it to the VM.
Create the managed disk from the VHD file:
{
"type": "Microsoft.Compute/disks",
"apiVersion": "2018-04-01",
"name": "[variables('diskName')]",
"location": "[parameters('location')]",
"properties": {
"creationData": {
"createOption": "Import",
"sourceUri": "[parameters('osDiskVhdUri')]"
},
"osType": "[parameters('osType')]"
}
}
Attach the managed disk to the VM:
"storageProfile": {
"osDisk": {
"osType": "[parameters('osType')]",
"createOption": "Attach",
"managedDisk": {
"id": "[resourceId('Microsoft.Compute/disks', variables('diskName'))]"
}
}
}
You can get the whole template here.
We have some Azure Table storage tables in our subscription and would like to migrate them to CosmosDB table API due to performance reasons. To do this, I started creating cosmos db account by selecting Table API but my deployment failed with the following error. When i tried with SQL API, it works.
{"code":"DeploymentFailed","message":"At least one resource deployment operation failed. Please list deployment operations for details. "details":[{"code":"BadRequest","message":"{\r\n \"code\": \"BadRequest\",\r\n \"message\": \"CORS rules are not supported for this API\rMicrosoft.Azure.Documents.Common/2.1.0.0\"\r\n}"}]}
Can someone please let me know what could be the reason for this?
#AngiSen, may be related to a recent (breaking) update of Azure Cosmos DB resource provider (Microsoft.DocumentDb/databaseAccounts) as I just noticed today (28th of Nov 2018) that a previously running deployment (as of 23th of Nov 2018) of Cosmos DB Table API is now failing with this same error:
9:16:23 AM - Resource Microsoft.DocumentDb/databaseAccounts 'xxx-xxx-xxx' failed with message '{
"code": "BadRequest",
"message": "CORS rules are not supported for this API\r\nActivityId: xxx, Microsoft.Azure.Documents.Common/2.1.0.0"
}'
In my case I'm using 2015-04-08 version with Table API but I don't configure explicitly the CORS part and anyway there's no such configuration option in the resource provider.
Digging into the existing Cosmos DB instance with https://resources.azure.com shows there's indeed a CORS member that is part of the definition:
{
"id": "/subscriptions/xxx/resourceGroups/xxx/providers/Microsoft.DocumentDB/databaseAccounts/xxx",
"name": "xxx",
"location": "North Europe",
"type": "Microsoft.DocumentDB/databaseAccounts",
"kind": "GlobalDocumentDB",
"tags": {},
"properties": {
"provisioningState": "Succeeded",
"documentEndpoint": "https://xxx.documents.azure.com:443/",
"tableEndpoint": "https://xxx.table.cosmosdb.azure.com:443/",
"ipRangeFilter": "",
"enableAutomaticFailover": false,
"enableMultipleWriteLocations": false,
"isVirtualNetworkFilterEnabled": false,
"virtualNetworkRules": [],
"EnabledApiTypes": "Table, Sql",
"databaseAccountOfferType": "Standard",
"consistencyPolicy": {
"defaultConsistencyLevel": "BoundedStaleness",
"maxIntervalInSeconds": 86400,
"maxStalenessPrefix": 1000000
},
"configurationOverrides": {},
"writeLocations": [
{
"id": "xxx-northeurope",
"locationName": "North Europe",
"documentEndpoint": "https://xxx-northeurope.documents.azure.com:443/",
"provisioningState": "Succeeded",
"failoverPriority": 0
}
],
"readLocations": [
{
"id": "xxx-northeurope",
"locationName": "North Europe",
"documentEndpoint": "https://xxx-northeurope.documents.azure.com:443/",
"provisioningState": "Succeeded",
"failoverPriority": 0
}
],
"locations": [
{
"id": "xxx-northeurope",
"locationName": "North Europe",
"documentEndpoint": "https://xxx-northeurope.documents.azure.com:443/",
"provisioningState": "Succeeded",
"failoverPriority": 0
}
],
"failoverPolicies": [
{
"id": "xxx-northeurope",
"locationName": "North Europe",
"failoverPriority": 0
}
],
"cors": [],
"capabilities": [
{
"name": "EnableTable"
}
]
}
}
Hope it'll get fixed quickly if it's indeed a breaking change...
Wanted to make an official statement here. I have spoken with the Cosmos DB team and they have a fix ready and it should be deployed tonight. Please let me know if you should have any questions. Thank you for posting the issue.