Deployment Agents in Azure VM Scale Set - azure

I am currently deploying a VM Scale Set (VMSS) using an ARM template which has a resource inside VMSS to install Azure extension for Azure DevOps (ADO) Deployment Agent. All is deployed successfully and a node is registered in ADO with all details as are in the ARM template. However the problem is that it installs the agent only on first node and (as far as I see) ignores the rest of the nodes. I've tested this with multiple nodes during creation of the scale set and with auto-scale as well. Both scenarios result in only first agent registered.
This is the code layout I'm using (I've removed the VMSS bits to reduce the template length here, there are of course OS, storage and network settings inside):
{
"type": "Microsoft.Compute/virtualMachineScaleSets",
"name": "[parameters('VMSSName')]",
"apiVersion": "2018-10-01",
"location": "[resourceGroup().location]",
"sku": {
"name": "[parameters('VMSSSize')]",
"capacity": "[parameters('VMSSCount')]",
"tier": "Standard"
},
"dependsOn": [],
"properties": {
"overprovision": "[variables('overProvision')]",
"upgradePolicy": {
"mode": "Automatic"
},
"virtualMachineProfile": {},
"storageProfile": {},
"networkProfile": {},
"extensionProfile": {
"extensions": [
{
"type": "Microsoft.Compute/virtualMachineScaleSets/extensions",
"name": "VMSS-NetworkWatcher",
"location": "[resourceGroup().location]",
"properties": {
"publisher": "Microsoft.Azure.NetworkWatcher",
"type": "[if(equals(parameters('Platform'), 'Windows'), 'NetworkWatcherAgentWindows', 'NetworkWatcherAgentLinux')]",
"typeHandlerVersion": "1.4",
"autoUpgradeMinorVersion": true
}
},
{
"type": "Microsoft.Compute/virtualMachineScaleSets/extensions",
"name": "VMSS-TeamServicesAgent",
"location": "[resourceGroup().location]",
"properties": {
"publisher": "Microsoft.VisualStudio.Services",
"type": "[if(equals(parameters('Platform'), 'Windows'), 'TeamServicesAgent', 'TeamServicesAgentLinux')]",
"typeHandlerVersion": "1.0",
"autoUpgradeMinorVersion": true,
"settings": {
"VSTSAccountName": "[parameters('VSTSAccountName')]",
"TeamProject": "[parameters('VSTSTeamProjectName')]",
"DeploymentGroup": "[parameters('VSTSDeploymentGroupName')]",
"AgentName": "[concat(parameters('VMSSName'),'-DG')]",
"Tags": "[parameters('VSTSDeploymentAgentTags')]"
},
"protectedSettings": {
"PATToken": "[parameters('VSTSPATToken')]"
}
}
}
]
}
}
}
}
Now the desired state, of course, is that all nodes will have agent installed so that I can use the Deployment Group inside Release pipeline.

your problem is in the fact that all agents have the same AgentName, so it effectively overwrites the agent and only the latest one "survives". I dont think there is anything you can do, unless you just amend the AgentName and it auto assigns based on computer name.
You can convert this to a script\dsc extension, that way you can calculate everything on the fly.

Related

Azure Function - What version of the ARM Template APIs support "Dynamic" YI tier sku for Linux?

Background Information
I have function app with 3 different functions inside. I also have written a powershell script that deploys these functions upstream to Azure portal. It basically just calls this function:
func azure functionapp publish $FUNCTION_APP.name --publish-local-settings -i --overwrite-settings -y
Whenever I run the powershell script, I see the following error message:
Setting PExtStorageQueue = ****
Syncing triggers...
Syncing triggers...
Syncing triggers...
Syncing triggers...
Syncing triggers...
Syncing triggers...
Error calling sync triggers (BadRequest). Request ID = 'b4a22cd4-5c5b-4b2b-a9d4-537cb9cdf96a'.
The functions do end up being deployed and I can trigger them. But none of them have URLs in the "Get URL" option.
What I've checked so far
Based on other posts, it seems that enabling slots might cause issues so I've verified that I DO NOT have slots enabled.
Container OS. I'm using Linux containers. The version of the Linux container is dotnet|3.1 for the function app. I've also tried to change it to "dotnetcore|6.0" per this article: https://medium.com/medialesson/solved-the-parameter-linuxfxversion-has-an-invalid-value-net6-linux-533c759456cd. But that just causes the script to die with the following error when I use dotnetcore:
Line |
38 | New-AzResourceGroupDeployment -ResourceGroupName $currentEnv.AZ_RESOU …
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| 3:05:27 PM - The deployment 'myresourcegroupname-resources' failed with error(s). Showing 1 out of 1 error(s). Status Message: The parameter LinuxFxVersion has an invalid value. (Code: BadRequest)
| - The parameter LinuxFxVersion has an invalid value. (Code:) - (Code:BadRequest) - (Code:) CorrelationId: 6f7e4efb-a0b9-46c1-9d4a-df8f6c8d155e
ARM Template for App Plan
Here's the section in the template for the app service plan:
{
"type": "Microsoft.Web/serverfarms",
"apiVersion": "2020-06-01",
"name": "[variables('appServicePlanPortalName')]",
"location": "[parameters('location')]",
"sku": {
"name": "Y1",
"tier": "Dynamic",
"size": "Y1",
"family": "Y",
"capacity": 0
},
"kind": "functionapp,linux",
"properties": {
"name": "[variables('appServicePlanPortalName')]",
"reserved": true,
"computeMode": "Dynamic"
}
},
ARM Template for the Function App
And here's a snippet from the ARM template showing you what I'm doing for the function app itself:
{
"type": "Microsoft.Web/sites",
"apiVersion": "2020-06-01",
"name": "[parameters('functionAppName')]",
"location": "[parameters('location')]",
"kind": "functionapp,linux",
"identity": {
"type": "UserAssigned",
"userAssignedIdentities": {
"[resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', variables('identityName'))]": {}
}
},
"dependsOn": [
"[resourceId('Microsoft.Web/serverfarms', variables('appServicePlanPortalName'))]",
"[resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', variables('identityName'))]"
],
"properties": {
"reserved": true,
"serverFarmId": "[resourceId('Microsoft.Web/serverfarms', variables('appServicePlanPortalName'))]",
"siteConfig": {
"linuxFxVersion": "dotnet|3.1",
"appSettings": [
{
"name": "APPINSIGHTS_INSTRUMENTATIONKEY",
"value": "[reference(resourceId('Microsoft.Insights/components', variables('appInsightsName')), '2015-05-01').InstrumentationKey]"
},
{
"name": "AzureWebJobsStorage",
"value": "[concat('DefaultEndpointsProtocol=https;AccountName=', parameters('storageAccountName'), ';EndpointSuffix=', environment().suffixes.storage, ';AccountKey=',listKeys(resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName')), '2019-06-01').keys[0].value)]"
},
{
"name": "FUNCTIONS_EXTENSION_VERSION",
"value": "~4"
},
{
"name": "FUNCTIONS_WORKER_RUNTIME",
"value": "dotnet"
}
]
}
}
}
Any help would be appreciated.
EDIT 1
In case it helps, I've been testing these changes to the ARM template: (git diff showing you what's been changed)
--- a/arm_templates/myresourcegroup-resources.json
+++ b/arm_templates/myresourcegroup-resources.json
## -168,17 +168,13 ##
"name": "[variables('appServicePlanPortalName')]",
"location": "[parameters('location')]",
"sku": {
- "name": "Y1",
- "tier": "Dynamic",
- "size": "Y1",
- "family": "Y",
- "capacity": 0
+ "tier": "Standard",
+ "name": "S1"
},
"kind": "functionapp,linux",
"properties": {
"name": "[variables('appServicePlanPortalName')]",
- "reserved": true,
- "computeMode": "Dynamic"
+ "reserved": true
}
},
{
## -201,7 +197,7 ##
"reserved": true,
"serverFarmId": "[resourceId('Microsoft.Web/serverfarms', variables('appServicePlanPortalName'))]",
"siteConfig": {
- "linuxFxVersion": "dotnet|3.1",
+ "linuxFxVersion": "DOTNETCORE|6.0",
"appSettings": [
{
"name": "APPINSIGHTS_INSTRUMENTATIONKEY",
It takes a super long time to publish but it eventually finishes.
It isn't able to list all my functions though, but I think I'm ok with that. This is what I see in the powershell script window:
Getting site publishing info...
Creating archive for current directory...
Uploading 13 MB [#################################################################################]
Upload completed successfully.
Deployment completed successfully.
App setting AzureWebJobsStorage is different between azure and local.settings.json
Overwriting setting in azure with local value because '--overwrite-settings [-y]' was specified.
Setting FUNCTIONS_WORKER_RUNTIME = ****
Setting PExtStorageTableName = ****
Setting PExtStorageQueue = ****
Functions in myresourcegroup-6z2ybl3tvqf6y-app:
PS C:\Users\me\src\azureTestDeploy>
But now I have URLs defined in the upstream functions!
Woot woot!
I'm going to revert each line again to try to narrow it down to the specific ARM change that solved the issue.
I'm guessing it'll be some combination, including the DOTNETCORE|6.0 change.
EDIT 2
I've been trying to play around with the different versions of the APIs to see how I can define a dynamic application service plan (Y1).
but I can't seem to get it working. I know our subscription supports Y1 in eastus for Windows. But I can't seem to get it working for Linux.
Anytime I use dynamic it gives me an error saying "The parameter LinuxFxVersion has an invalid value. (Code: BadRequest)"

ARM template for Azure Function ignores preWarmedInstanceCount setting

I'm deploying Azure function into Premium Function Plan (Elastic) using Azure powershell script:
New-AzResourceGroupDeployment -ResourceGroupName $RESOURCE_GROUP -TemplateFile "function-app.json" -TemplateParameterObject $params -Name $APP_SERVICE_NAME -Mode Incremental > $null
And deployment ignores my preWarmedInstanceCount setting. Newly created function has Always Ready Instances = 0 (see screenshot)
ARM template of function:
{
"apiVersion": "2020-06-01",
"name": "[parameters('siteName')]",
"type": "Microsoft.Web/sites",
"identity": {
"type": "systemAssigned"
},
"kind": "functionapp",
"location": "[resourceGroup().location]",
"properties": {
"name": "[parameters('siteName')]",
"serverFarmId": "[resourceId(parameters('appServicePlanRg'),'Microsoft.Web/serverfarms',parameters('appServicePlanName'))]",
"clientAffinityEnabled": false,
"siteConfig": {
"use32BitWorkerProcess": false,
"preWarmedInstanceCount": 2,
"appSettings": [
{
"name": "FUNCTIONS_EXTENSION_VERSION",
"value": "~3"
},
{
"name": "WEBSITE_CONTENTAZUREFILECONNECTIONSTRING",
"value": "DefaultEndpointsProtocol=https;AccountName=xxx..."
},
{
"name": "WEBSITE_CONTENTSHARE",
"value": "asggas"
}
]
}
}
}
It seems to me few days ago it did worked properly and I managed to set that value via arm template and now I can only update it via Azure portal.
Here is ARM template of my hosting plan:
{
"type": "Microsoft.Web/serverfarms",
"apiVersion": "2018-02-01",
"name": "[parameters('appServicePlanName')]",
"location": "[resourceGroup().location]",
"properties": {
"name": "[parameters('appServicePlanName')]",
"workerSize": "1",
"numberOfWorkers": "1",
"maximumElasticWorkerCount": 20
},
"sku": {
"Tier": "ElasticPremium",
"Name": "EP2"
}
}
After some investigation I've come to "Activity Log" (after manual update on portal) screen and was surprised that my desired property is named as "minimumElasticInstanceCount" (it is not documented anywhere in either ARM API version https://learn.microsoft.com/en-us/azure/templates/microsoft.web/sites)
Then I added this field to my ARM template and all looks good for now. Also some explanation and difference between "Always Ready Instances"(minimumElasticInstanceCount) and "Pre-warmed instances"(preWarmedInstanceCount) are posted here: https://learn.microsoft.com/uk-ua/azure/azure-functions/functions-premium-plan
So in Azure Portal Pre-warmed instance setting is not displayed. And I was looking for another setting.
I have tried to replicate the issue and was able to figure out the issue here.
Instead of "preWarmedInstanceCount" try "reservedInstanceCount".
You can check from the Azure portal itself. Once you have updated the reserved instance count from the portal you can export the template :
Then compare your previous template with the exported one, you will be able to see the difference.

Azure ARM - Microsoft.Resources/deploymentScripts

I am going to configure created azure VM (for example, install role, initialize new hdd disk etc). I see that there is a new feature Microsoft.Resources/deploymentScripts in azure ARM. As per documantation I created Managed Identity in my subscription, give Contributor permissions to newly created Managed Identity, on Subscription level. then I developed below ARM template using Microsoft.Resources/deploymentScripts feature. code pasted below. I want to paste this code into my ARM template for VM deployment.Question is if I will be able to use this approach to perform scripts like: installing role on the OS level, like IIS or WSUS, configure HDD etc...
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"name": {
"type": "string",
"defaultValue": "'John Dole'"
},
"utcValue": {
"type": "string",
"defaultValue": "[utcNow()]"
}
},
"resources": [
{
"type": "Microsoft.Resources/deploymentScripts",
"apiVersion": "2019-10-01-preview",
"name": "runPowerShellInlineWithOutput",
"location": "westeurope",
"kind": "AzurePowerShell",
"identity": {
"type": "userAssigned",
"userAssignedIdentities": {
"/subscriptions/SubID/resourceGroups/RGname/providers/Microsoft.ManagedIdentity/userAssignedIdentities/MI-ARMdeployment": {}
}
},
"properties": {
"forceUpdateTag": "[parameters('utcValue')]",
"azPowerShellVersion": "3.0",
"scriptContent": "
$output = 'hello'
Write-Output $output",
"arguments": "",
"timeout": "PT1H",
"cleanupPreference": "OnSuccess",
"retentionInterval": "P1D"
}
}
]
}
Well, yes (with some hacks), but its not meant for that. Its meant for provisioning\configuring Azure level resources, not things inside of the VM.
You have DSC extension and script extension for that (available for both windows\linux).

Azure ARM template depend on resources in copy loop

I am creating ARM template which takes in hash table of subnets and creates those. However, it looks that I need to wait for the first subnet to be ready before creating the second etc. But I do not know how I could depend on the previous subnet in copy loop. My template resource looks like this currently:
{
"apiVersion": "2018-06-01",
"type": "Microsoft.Network/virtualNetworks/subnets",
"name": "[concat(parameters('vnetName') , '/' , parameters('subnets').settings[copyIndex()].name)]",
"location": "[variables('location')]",
"copy": {
"name": "subnetLoop",
"count": "[variables('subnetcount')]"
},
"dependsOn": ["[parameters('vnetName')]",
"[resourceId(variables('rgname'), 'Microsoft.Network/virtualNetworks/subnets', parameters('vNetName'), parameters('subnets').settings[copyIndex()].name)]"
],
"properties": {
"addressPrefix": "[parameters('subnets').settings[copyIndex()].addressPrefix]",
}
Which does not work because the first subnet cannot reference itself.
you can use "mode": "serial" to workaround that.
"copy": {
"name": "subnetLoop",
"count": "[variables('subnetcount')]",
"mode": "serial"
},
https://learn.microsoft.com/en-us/azure/azure-resource-manager/resource-group-create-multiple#resource-iteration
but you really need to look at properties loop, check this link:
https://learn.microsoft.com/en-us/azure/architecture/building-blocks/extending-templates/objects-as-parameters#using-a-property-object-in-a-copy-loop

Deploy nested resources separately

Due to separation of duty I need to split an existing ARM template into two single templates - one for the resource and one for the logging
the original templates looks like this:
"resources": [
{ // https://learn.microsoft.com/en-us/azure/templates/microsoft.datafactory/factories
"type": "Microsoft.DataFactory/factories",
"name": "[variables('dataFactoryName')]",
"apiVersion": "[variables('apiVersion')]",
"location": "[resourceGroup().location]",
"tags": {},
"identity": {
"type": "SystemAssigned"
},
"properties": {},
"resources": [
{
"type": "providers/diagnosticSettings",
"name": "[concat('Microsoft.Insights/', variables('logSettingName'))]",
"dependsOn": [
"[resourceId('Microsoft.DataFactory/factories', variables('dataFactoryName'))]"
],
"apiVersion": "2017-05-01-preview",
"location": "[resourceGroup().location]",
"tags": {},
"properties": {
"name": "[variables('logSettingName')]",
"workspaceId": "[concat('/subscriptions/', subscription().subscriptionId, '/resourceGroups/', parameters('logAnalyticsObject').resourceGroup, '//providers/Microsoft.OperationalInsights/workspaces/', parameters('logAnalyticsObject').name)]",
"logs": "[parameters('logAnalyticsObject').adfv2.logs]",
"metrics": "[parameters('logAnalyticsObject').adfv2.metrics]"
}
}
]
}
The first part is quite easy, I just remove the sub-resource but how to get the second part (resource with "type": "providers/diagnosticSettings") correctly so it can be deployed from a different template?
Is this possible at all or are these strongly tied together?
I already tried different things like
"type": "Microsoft.DataFactory/factories/providers/diagnosticSettings",
"name": "[concat('Microsoft.Insights/', variables('name'))]",
but ended up with error messages like this:
Unable to process template language expressions for resource '/subscriptions/fb1e20c4-0878-4949-ac10-f92a9ac35db4/resourceGroups/swe-sdp-dv0
-rgp-adp/providers/Microsoft.Resources/deployments/DataFactory_LogAnalytics_Resource' at line '67' and column '5'. 'Unable to evaluate template language
function 'resourceId': function requires exactly one multi-segmented argument which must be resource type including resource provider namespace. Current
function arguments 'fb1e20c4-0878-4949-ac10-f92a9ac35db4,swe-sdp-dv0-rgp-anl,Microsoft.Insights,swe-sdp-dv0-oms-001'. Please see
https://aka.ms/arm-template-expressions/#resourceid for usage details.
I think to make it work I would need the right combination of "type", "name" and probably also "dependsOn"
ok, according to this, you would need to do this:
"type": "Microsoft.DataFactory/factories/providers/diagnosticSettings",
"name": "[concat(variables('dataFactoryName'), '/Microsoft.Insights/', variables('name'))]",
you dont need dependsOn, because resources are under different templates.

Resources