Azure: Cannot pass or execute a custom data to VM - linux

I want to deploy an Azure Ubuntu 18.04-LTS VM with a custom data file during an automation test, using tmplate.json and parameters.json files.
Although, the VM was deployed successfully, It seem that the custom data execution have failed and I do not understand why...
According to this link, cloud-init is available in the image that I use.
My template.json file contain:
"parameters": {
...
"customData": {
"type": "string"
}
...
},
"resources": [
...
{
"type": "Microsoft.Compute/virtualMachines",
"apiVersion": "2020-06-01",
"name": "[parameters('virtualMachineName')]",
"location": "[parameters('location')]",
"dependsOn": [
"[concat('Microsoft.Network/networkInterfaces/', parameters('networkInterfaceName'))]"
],
"properties": {
...
"osProfile": {
"computerName": "[parameters('virtualMachineComputerName')]",
"adminUsername": "[parameters('adminUsername')]",
"linuxConfiguration": {
"disablePasswordAuthentication": true,
"ssh": {
"publicKeys": [
{
"path": "[concat('/home/', parameters('adminUsername'), '/.ssh/authorized_keys')]",
"keyData": "[parameters('adminPublicKey')]"
}
]
}
},
"customData": "[parameters('customData')]"
},
...
}
]
Also, I have a parameters.json file which contain:
"parameters": {
...
"customData": {
"value": "I2Nsb3VkLWNvbmZpZ1xucGFja2FnZV91cGdyYWRlOiB0cnVlXG5wYWNrYWdlczpcbiAgLSBpcGVyZlxuICAtIHRjcHJlcGxheVxuICAtIG5ldHBlcmY="
}
}
The value is YAML base64 encoded, and it's decoded text is:
#cloud-config\npackage_upgrade: true\npackages:\n - iperf\n - tcpreplay\n - netperf
(*) the above template.json and parameters.json files are partial - I omitted what seems to be not related to my problem.
I also checked the file /var/log/waagent.log and I did not found anything suspicious..
However, after I deployed the ubuntu VM manually and pasted the above YAML in the custom data section (in the advance setting page) every worked fine ('iperf', 'netperf' and 'tcpreplay' commands were found)
Any help will be appreciated!!

According to my experience, the problem is that the value for the custom data is not right. I check the VM that the cloud-init provision successfully, the code does not match yours. You can check the file /var/lib/waagent/ovf-env.xml yourself. Do not change the text yourself into a string. You can encode the text online.

Related

arm template virtualNetworkName creation appendix issue

I am trying to get a arm template running and have hit an issue with the virtualnetwork creation.
azuredeploy.json
"virtualNetworkName": {
"type": "string",
"metadata": {
"description": "Name of virtual network to be created"
},
"defaultValue": "autohav2VNET"
},
vnet-net.json
"resources": [
{
"name": "[parameters('virtualNetworkName')]",
"type": "Microsoft.Network/virtualNetworks",
"location": "[parameters('location')]",
"apiVersion": "2015-06-15",
"properties": {
"addressSpace": {
"addressPrefixes": [
"[parameters('virtualNetworkAddressRange')]"
]
},
"subnets": "[parameters('subnets')]"
}
}
]
The issue I am getting is that the vnet gets created with an appendix such as this: autohav2VNETl5g
So when this gets used to create a loadblancer, the names doe not match the defined parameter and the creation fails.
..../virtualNetworks/AUTOHAV2VNET referenced by resource .... /Microsoft.Network/loadBalancers/sqlLoadBalancer was not found.
Any suggestions?
with the data given it impossible to be sure why this is happening. you are probably passing in a value to the parameter virtualNetworkName. because if you wouldn't, than the vnet name would be: autohav2VNET.
ARM templates do not append anything anywhere just because they are arm templates. they only do what you designed them to do.
to help with debugging: how you are invoking the template and full template + full parameters file.

ARM When using reference() in an copy iterator, arm template parser seems to break

I have written a ARM template that dynamically builds out the app settings based on a JSON object parameter. This allows for adding any app setting without having to modify the template:
parameters.json
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"app_settings": {
"value": {
"keyvalue_pairs": [
{
"name": "appsetting1",
"value": "value"
},
{
"name": "ApplicationInsights:InstrumentationKey",
"value": ""
}
]
}
}
}
}
chopped down working template.json file
"resources": [
{
"comments": "",
"type": "microsoft.insights/components",
"kind": "web",
"name": "[variables('app_service_name')]",
"apiVersion": "2014-04-01",
"location": "[resourceGroup().location]",
"tags": {
"[variables('hiddenlink_app_service')]": "Resource"
},
"scale": null,
"properties": {
"ApplicationId": "[variables('app_service_name')]"
}
},
{
"apiVersion": "2015-08-01",
"name": "[concat(variables('app_service_name'),'/stage-slot')]",
"type": "Microsoft.Web/sites/slots",
"tags": {
"displayName": "stage-slot"
},
"location": "[resourceGroup().location]",
"dependsOn": [
"[resourceId('Microsoft.Web/Sites', variables('app_service_name'))]"
],
"properties": {
"clientAffinityEnabled": false,
"siteConfig": {
"copy": [
{
"name": "appSettings",
"count": "[length(parameters('app_settings').keyvalue_pairs)]",
"input": {
"name": "[parameters('app_settings').keyvalue_pairs[copyIndex('appSettings')].name]",
"value": "[parameters('app_settings').keyvalue_pairs[copyIndex('appSettings')].value]"
}
}
]
}
}
}
I now try and conditionally reference the app insights instrumentation key on my app settings hoping to override the app insights instrumentationkey from the resource.
"siteConfig": {
"copy": [
{
"name": "appSettings",
"count": "[length(parameters('app_settings').keyvalue_pairs)]",
"input": {
"name": "[parameters('app_settings').keyvalue_pairs[copyIndex('appSettings')].name]",
"value": "[if(equals(parameters('app_settings').keyvalue_pairs[copyIndex('appSettings')].name,'ApplicationInsights:InstrumentationKey'),reference(variables('appInsightsResource')).InstrumentationKey,parameters('app_settings').keyvalue_pairs[copyIndex('appSettings')].value)]"
}
}
]
}
This started throwing errors saying the if statement needs a boolean first parameter, but I didnt see anything wrong with it so I tried the following snippet and it worked so it leads me to believe that the use of the "reference()" inside a conditional isnt valid:
"siteConfig": {
"copy": [
{
"name": "appSettings",
"count": "[length(parameters('app_settings').keyvalue_pairs)]",
"input": {
"name": "[parameters('app_settings').keyvalue_pairs[copyIndex('appSettings')].name]",
"value": "[if(equals(parameters('app_settings').keyvalue_pairs[copyIndex('appSettings')].name,'ApplicationInsights:InstrumentationKey'),'testvalue',parameters('app_settings').keyvalue_pairs[copyIndex('appSettings')].value)]"
}
}
]
}
Additionally if I remove the whole "if()" and explicitly put "reference(variables('appInsightsResource')).InstrumentationKey" into the value, it outputs the right value so I know that this reference() call works but seems to break down when added inside an "if()" conditional statement.
The question is, is there any way to get this to work? I am trying to dynamically set the Instrumentation key while keeping in tact the ability to pass in a JSON object for my app settings
I'm seeing extremely confusing results with copy properties construct in general. most of the times is blows up with absolutely cryptic errors.
You won’t be able to use the reference() function in the count property – the copy loop is expanded at compile time – reference is evaluated at run-time. also, today I cant reproduce my working example, but i had a working example of reference function working in a copy properties, without if() though. You might want to create a bare minimum example showing how this doesnt work (so only 1 resource ideally). If that doesnt work you might want to raise an issue on github and\or azure feedback
You might be able to work around this by using nested deployments. but generally using properties copy in a clever way is a bit of a pain, due to compile\runtime issues.

Unable to build string in ARM template

I am currently working on building a few templates for some of the basic deployments that I use daily and have them working except for the last piece. The issue is that i am using t-shirt sizing to choose a specific vhd as I am deploying and I can not see to get the syntax correct for the uri string.
I know the syntax to call the variable on its own but when I add it into the uri string, it fails.
[variables(‘vhd’)[parameters('version')]]
Parameter:
"version": {
"type": "string",
"defaultValue": "10.5",
"allowedValues": [
"10.3.1",
"10.4.1",
"10.5"
],
Variable:
"vhd": {
"10.3.1": "20170524144905.vhd",
"10.4.1": "20170524144656.vhd",
"10.5": "20170524133408.vhd"
},
String:
"vhd": {
"uri": "[concat(concat(reference(resourceId(parameters('virtualMachineName'), 'Microsoft.Storage/storageAccounts', variables('storageAccountName')), '2015-06-15').primaryEndpoints['blob'], 'vhds/'), parameters('virtualMachineName'), '20170524133408.vhd')]"
},

How to Copy Azure SQL Database using ARM Template

Not sure if it is supported in ARM. I could find power-shell references only.
You cannot currently deploy a dacpac with an ARM template. The link above uses PowerShell but not ARM. You can create however create a database from a source database as a copy using an ARM template.
A simple way to find an example template for any Azure action is to perform the action in the portal - in this case, copy a database - and then open the appropriate resource group blade in the portal, list the deployments, locate the deployment just submitted and open it. Then select ViewTemplate from the menu bar and examine both the Template tab and the Parameters tab. These show you the full template and the parameter values actually used. You can then download the template, with accompanying Powershell script.
For database copy, here is the template:
{
"$schema": "http://schema.management.azure.com/schemas/2014-04-01-preview/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"databaseName": {
"type": "string"
},
"serverName": {
"type": "string"
},
"location": {
"type": "string"
},
"createMode": {
"type": "string"
},
"sourceDatabaseId": {
"type": "string"
},
"requestedServiceObjectiveName": {
"type": "string"
}
},
"resources": [
{
"apiVersion": "2014-04-01-preview",
"location": "[parameters('location')]",
"name": "[concat(parameters('serverName'), '/', parameters('databaseName'))]",
"properties": {
"createMode": "[parameters('createMode')]",
"sourceDatabaseId": "[parameters('sourceDatabaseId')]",
"requestedServiceObjectiveName": "[parameters('requestedServiceObjectiveName')]"
},
"type": "Microsoft.Sql/servers/databases"
}
]
}
For database copy createMode = 'Copy'
And be sure to provide a fully qualified resourceId formatted as follows:
"/subscriptions/<sub-id>/resourceGroups/<resourceGroupName>/providers/Microsoft.Sql/Servers/<server-name>/databases/<database-name>"
Make sure the resource group name capitalization is correct and that the server name is all lower case.
You can use the sourceDatabaseId property to reference another database. Then you can specify various createModes depending on what type of database you would like to create:
{
"properties": {
"createMode": "OnlineSecondary",
"sourceDatabaseId": "[resourceId('Microsoft.Sql/servers/databases', variables('sql01Name'), 'databasename')]"
}
}
http://msdn.microsoft.com/en-us/library/azure/mt163685.aspx
The answer above from #Bill Gibson - MSFT works if you are using a Microsoft.Sql/servers resource, however if you're using a Microsoft.Sql/managedInstances resource you'll need to use the appropriate Microsoft.Sql/managedInstance/databases - ARM Template.
The following works for me to perform a PointInTimeRestore accessing a source database that lives in another resource group (the variables and parameters are left as an exercise to the reader):
{
"type": "Microsoft.Sql/managedInstances/databases",
"name": "[concat(variables('destinationSqlManagedInstanceName'), '/', 'AdventureWorks')]",
"apiVersion": "2021-11-01",
"location": "[parameters('location')]",
"properties": {
"createMode": "PointInTimeRestore",
"restorePointInTime": "2022-12-14T12:00:00Z",
"sourceDatabaseId": "[resourceId(variables('sourceResourceGroupName'), 'Microsoft.Sql/managedInstances/databases', variables('sourceSqlManagedInstanceName'), 'AdventureWorks')]"
}
}
The documentation is broken in a few ways:
When attempting to perform a PointInTimeRestore the properties referenced (SourceDatabaseName, SourceManagedInstanceName, PointInTime) do not exist. Rather the following properties are used: restorePointInTime and sourceDatabaseId which are documented in the documentation.
Additionally, the restorePointInTime indicates that the time should be in ISO8601 format, however this is not the same as what is returned by utcNow(). Testing has shown that you must provide it in this version of the ISO8601 format: yyyy-MM-ddTHH:mm:ssZ which can be done using utcNow('yyyy-MM-ddTHH:mm:ssZ').
I have created an issue to try and get the documentation fixed up here: https://github.com/MicrosoftDocs/azure-docs/issues/102717

Azure Automation DSC compilation job failing with 404 from ARM template

I have created an ARM template which successfully creates an Automation Account in Azure and then creates a module and DSC configuration in that account.
When I add a Microsoft.Automation/automationAccounts/Compilationjobs resource to compile the DSC configuration, the template deployment fails at this step with 404 - File or directory not found.
The Compilationjobs resource exists as a top level resource in the template as follows:
{
"apiVersion": "2015-10-31",
"type": "Microsoft.Automation/automationAccounts/Compilationjobs",
"name": "automationAccountName/jobId123",
"location": "[variables('location')]",
"tags": {
},
"dependsOn": [
"Microsoft.Automation/automationAccounts/automationAccountName",
"modulesResourceLoop"
],
"properties": {
"configuration": {
"name": "DSCConfigurationName"
}
}
}
When I call Start-AzureRmAutomationDscCompilationJob with the same details the compilation job is created and completes successfully.
Compiling the configuration involves creating a compliationJob. Under the hood it's a PUT call to /CompiliationJobs/{guid}. so the trick here is to pass a new guid into the arm template when invoking compilation job.
Something like the following, you will need to define the parameter compilationJobGuid:
{
"name": "[parameters('compilationJobGuid')]",
"apiVersion": "2015-10-31",
"type": "Microsoft.Automation/automationAccounts/Compilationjobs",
"location": "[variables('location')]",
"tags": {
},
"dependsOn": [
"Microsoft.Automation/automationAccounts/automationAccountName",
"modulesResourceLoop"
],
"properties": {
"configuration": {
"name": "DSCConfigurationName"
}
}
}

Resources