I am attempting to deploy a new Azure Virtual Network Gateway to an existing VNET that includes several subnets. I am configuring this in a test environment first with a dummy subnet. I am using ARM to create a .json template and parameters file, which I am deploying via Jenkins. Currently the template attempts to redeploy the whole VNET when it deploys the Virtual Network Gateway. I do not want it to do this. I want it to deploy the Virtual Network Gateway to the existing VNET. Please see below for how I am coding the VNET in the template.
{
"apiVersion": "2019-04-01",
"type": "Microsoft.Network/virtualNetworks",
"name": "[parameters('virtualNetworkName')]",
"location": "[resourceGroup().location]",
"properties": {
"addressSpace": {
"addressPrefixes": [
"[parameters('azureVNetAddressPrefix')]"
]
},
"subnets": [
{
"name": "GatewaySubnet",
"properties": {
"addressPrefix": "[parameters('gatewaySubnetPrefix')]"
}
}
]
}
}
I am getting the following error in Jenkins when deploying this template:
"code": "InUseSubnetCannotBeDeleted",
"message": "Subnet testing-subnet is in use by /subscriptions/****/resourceGroups/networks-dev-rg/providers/Microsoft.Network/networkInterfaces/dev-jmp-d31653/ipConfigurations/ipconfig1 and cannot be deleted. In order to delete the subnet, delete all the resources within the subnet. See aka.ms/deletesubnet."
I've looked at the Microsoft knowledgebase but I've struggled to find an explanation of how I can do this, or whether it's even possible. Ideally, I'd like to avoid listing all of the subnets in the vnet, as this is a template I want to apply to different vnets with different subnets.
Can anyone provide answers or advice? Thanks.
Unfortunately, this does not seem to be supported very well in ARM. This is because a VNET is a resource and a subnet is a property of that resource. When an ARM template is deployed, any resources not mentioned are ignored (in iterative mode, at least).
However, properties of existing resources that are mentioned MUST BE SPECIFIED. This is because Azure tries to implement the resource as specified in the template. If a property is different, it will alter it. If a property is absent, it will REMOVE it.
Potential solutions:
Have multiple templates for each of your vnets. When you make a change, you update the whole vnet. This requires you to track several templates and is not ideal for infrastructure as code, but is a simple solution.
Use a powershell solution instead:
https://learn.microsoft.com/en-us/azure/vpn-gateway/vpn-gateway-tutorial-create-gateway-powershell. I haven't tried this myself as I've been told to use ARM by my superiors, but it has been suggested on several forums as an alternative.
You could also attempt to use a copyloop as per this guidance, but this has limited utility and I haven't yet verified if you can use a name array rather than a number array:
https://pkm-technology.com/azure-vnet-json/
Update your subnets as part of a separate template. This requires you to also update your master vnet template as well, otherwise your new subnets will be removed if you ever redeploy the master vnet template. Also, you can only add subnets in this way. It doesn't help if you want to do something else, such as deploy a VPN gateway.
The following ARM template will add a subnet to a virtual network with existing subnets and will not disturb the existing subnets.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"virtualNetworkName": {
"type": "string",
"defaultValue": "VNet1"
},
"gatewaySubnetPrefix": {
"type": "string",
"defaultValue": "10.0.2.0/24"
}
},
"variables": {},
"resources": [
{
"apiVersion": "2019-04-01",
"type": "Microsoft.Network/virtualNetworks/subnets",
"name": "[concat(parameters('virtualNetworkName'), '/GatewaySubnet')]",
"location": "[resourceGroup().location]",
"properties": {
"addressPrefix": "[parameters('gatewaySubnetPrefix')]"
}
}
]
}
Related
Since it is not possible to create an organization with the DevOps Rest API and the SDK for organizations is still in preview and not functional, we currently try to programmatically create an organization with an ARM template.
In the Azure Portal, it is possible to create a new organization with an ARM template like this:
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/Microsoft.Resources.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"organizationName": {
"type": "string",
"defaultValue": ""
},
"organizationIdentifier": {
"type": "string",
"defaultValue": ""
},
"administrator": {
"type": "string",
"defaultValue": ""
}
},
"resources": [
{
"name": "[parameters('organizationIdentifier')]",
"type": "microsoft.visualstudio/account",
"location": "West Europe",
"description": "[parameters('organizationName')]",
"apiVersion": "2014-02-26",
"properties": {
"operationType": "Create",
"accountName": "[parameters('organizationIdentifier')]",
"ownerUpn": "[parameters('administrator')]"
}
}
]
}
Unfortunately we couldn't find a solution to create this within a C# Azure Function programmatically, since the SDK would want a resource group, which doesn't make sense in this context. Is there a way to do this or is it simply not possible to create an organization automatically at this point?
Yes, as of now there is no ways to create an DevOps organization using REST API.
The way to create Azure DevOps Organization is Manual creation or using ARM Template.
Using ARM Template, you have to specify the Resource name as a parameter. That is also we need to mention the Organization Name before deploying the ARM Template.
As of now the REST API is available for some of the services.
References
-Automating organization and project creation in Azure DevOps Blog gives the clear view
I have created an instance of Azure Kubernetes Service (AKS) and have discovered that apart from the resource group I created the AKS instance in, two other resource groups were created for me. Here is what my resource groups and their contents looks like:
MyResourceGroup-Production
MyAKSInstance - Azure Kubernetes Service (AKS)
DefaultResourceGroup-WEU
ContainerInsights(MyAKSInstance) - Solution
MyAKSInstance - Log Analytics
MC_MyResourceGroup-Production_MyAKSInstance_westeurope
agentpool-availabilitySet-36219400 - Availability set
aks-agentpool-36219400-0 - Virtual machine
aks-agentpool-36219400-0_OsDisk_1_09469b24b1ff4526bcfd5d00840cfbbc - Disk
aks-agentpool-36219400-nic-0 - Network interface
aks-agentpool-36219400-nsg - Network security group
aks-agentpool-36219400-routetable - Route table
aks-vnet-36219400 - Virtual network
I have a few questions about these two separate resource groups:
Can I rename the resource groups or control how they are named from my ARM template in the first place at the time of creation?
Can I move the contents of DefaultResourceGroup-WEU into MyResourceGroup-Production?
Can I safely edit their settings?
The DefaultResourceGroup-WEU seems to be created if you enable Log Analytics. Can I use this instance for accepting logs from other instances?
UPDATE
I managed to pre-create a log analytics resource and use that for Kubernetes. However, there is a third resource that I'm having trouble moving into my resource group:
{
"type": "Microsoft.Resources/deployments",
"name": "SolutionDeployment",
"apiVersion": "2017-05-10",
"resourceGroup": "[split(parameters('omsWorkspaceId'),'/')[4]]",
"subscriptionId": "[split(parameters('omsWorkspaceId'),'/')[2]]",
"properties": {
"mode": "Incremental",
"template": {
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {},
"variables": {},
"resources": [
{
"apiVersion": "2015-11-01-preview",
"type": "Microsoft.OperationsManagement/solutions",
"location": "[parameters('workspaceRegion')]",
"name": "[concat('ContainerInsights', '(', split(parameters('omsWorkspaceId'),'/')[8], ')')]",
"properties": {
"workspaceResourceId": "[parameters('omsWorkspaceId')]"
},
"plan": {
"name": "[concat('ContainerInsights', '(', split(parameters('omsWorkspaceId'),'/')[8], ')')]",
"product": "[concat('OMSGallery/', 'ContainerInsights')]",
"promotionCode": "",
"publisher": "Microsoft"
}
}
]
}
},
"dependsOn": []
}
No, you cant.
Yes, but I'd advice against it. I'd advice remove the health metrics from AKS, delete that resource group, create OMS in the same resource group with AKS (or wherever you need your OMS to be) and then use that OMS. it will just create container solution for you in the same resource group where oms is in.
To extent, if you break anything AKS wont fix it
Yes you can, but you better rework it like I mention in point 2.
How can I deploy my function app attached to a VNet using an arm template?
Right now my arm template deploys an AppServicePlan based function app just fine and sets the "vnetName" in the "site" resource "properties" section...but after the deploy the function app still shows as "not configured" for any VNet. I can then go into the portal and add to the desired VNet in a couple of clicks...but why doesn't it work via the arm template? Can someone please provide a sample arm template to do this?
Upon further review this is not possible via only ARM. Here are the instructions to connect a web app to a VNet: https://learn.microsoft.com/en-gb/azure/app-service-web/app-service-vnet-integration-powershell
Old Answer: Here is another post trying to achieve the same with Web Apps (Functions is built on Web Apps): https://stackoverflow.com/a/39518349/5915331
If I had to guess based on the powershell command, try adding a sub resource to the site of type config with name web, similar to the arm template for a github deployment: https://github.com/MicrosoftDocs/azure-docs/blob/master/articles/app-service-web/app-service-web-arm-from-github-provision.md#web-app
There may be some magic happening under the hood, however.
{
"name": "[parameters('siteName')]",
"type": "Microsoft.Web/sites"
"location": ...,
"apiVersion": ...,
"properties": ...,
"resources": [
"name": "web",
"type": "config",
"apiVersion": "2015-08-01",
"dependsOn": [
"[resourceId('Microsoft.Web/Sites', parameters('siteName'))]"
],
"properties": {
"vnetName": "[parameters('vnetName')]"
},
]
}
I can do everything with ARM except create the VPN package - which is what makes the difference between just setting the resource properties and actually making the app connect to the VNET correctly.
Everything in this article (https://learn.microsoft.com/en-gb/azure/app-service-web/app-service-vnet-integration-powershell) can be easily converted from PowerShell to ARM except for Get-AzureRmVpnClientPackage
Hypothetically, if you are using a legacy VNET, you could get the VPN client package URL via ARM because the legacy VNET resource provider supports that operation (https://learn.microsoft.com/en-us/azure/active-directory/role-based-access-control-resource-provider-operations):
Microsoft.ClassicNetwork/virtualNetworks/gateways/listPackage/action
The ARM VNET provider does not seem to
I'm trying to learn Azure Resource Templates and am trying to understand the workflow behind when to use them and when to use the REST API.
My sense is that creating a Virtual Network and Subnets in Azure is a fairly uncommon occurance, once you get that set up as you want you don't modify it too frequently, you deploy things into that structure.
So with regard to an ARM Template let's say I have a template with resources for VNET and Subnet. To take the example from https://learn.microsoft.com/en-us/azure/azure-resource-manager/resource-manager-template-walkthrough#virtual-network-and-subnet I might have:
{
"apiVersion": "2015-06-15",
"type": "Microsoft.Network/virtualNetworks",
"name": "[parameters('vnetName')]",
"location": "[resourceGroup().location]",
"properties": {
"addressSpace": {
"addressPrefixes": [
"10.0.0.0/16"
]
},
"subnets": [
{
"name": "[variables('subnetName')]",
"properties": {
"addressPrefix": "10.0.0.0/24"
}
}
]
}
}
which I deploy to a Resource Group. Let's say I then add a Load Balancer and redeploy the template. In this case the user then gets asked to supply the value for the vnetName parameter again and of course cannot supply the same value so we would end up with another VNET which is not what we want.
So is the workflow that you define your ARM Template (VNET, LBs, Subnets, NICs etc) in one go and then deploy? Then when you want to deploy VMs, Scale Sets etc you use the REST API to deploy then to the Resource Group / VNET Subnet? Or is there a way to incrementally build up an ARM Template and deploy it numerous times in a way that if a VNET already exists (for example) the user is not prompted to supply details for another one?
I've read around and seen incremental mode (default unless complete is specified) but not sure if this is relevant and if it is how to use it.
Many thanks for any help!
Update
OK so I can now use azure group deployment create -f azuredeploy.json -g ARM-Template-Tests -m Incremental and have modified the VNET resource in my template from
{
"apiVersion": "2016-09-01",
"type": "Microsoft.Network/virtualNetworks",
"name": "[variables('virtualNetworkName')]",
"location": "[resourceGroup().location]",
"properties": {
"addressSpace": {
"addressPrefixes": [
"[variables('addressPrefix')]"
]
},
"subnets": [
{
"name": "[variables('subnetName')]",
"properties": {
"addressPrefix": "[variables('subnetPrefix')]"
}
}
]
}
},
to
{
"apiVersion": "2015-05-01-preview",
"type": "Microsoft.Network/virtualNetworks",
"name": "[parameters('virtualNetworkName')]",
"location": "[resourceGroup().location]",
"properties": {
"addressSpace": {
"addressPrefixes": [
"[parameters('addressPrefix')]"
]
},
"subnets": [
{
"name": "[parameters('subnet1Name')]",
"properties": {
"addressPrefix": "[parameters('subnet1Prefix')]"
}
},
{
"name": "[parameters('gatewaySubnet')]",
"properties": {
"addressPrefix": "[parameters('gatewaySubnetPrefix')]"
}
}
]
}
},
but the subnets don't change. Should they using azure group deployment create -f azuredeploy.json -g ARM-Template-Tests -m Incremental
I am going to piggy back on this Azure documentation. Referencing appropriate section below:
Incremental and complete deployments
When deploying your resources,
you specify that the deployment is either an incremental update or a
complete update. By default, Resource Manager handles deployments as
incremental updates to the resource group.
With incremental deployment, Resource Manager
leaves unchanged resources that exist in the resource group but are not specified in the template
adds resources that are specified in the template but do not exist in the resource group
does not reprovision resources that exist in the resource group in the same condition defined in the template
reprovisions existing resources that have updated settings in the template
With complete deployment, Resource Manager:
deletes resources that exist in the resource group but are not specified in the template
adds resources that are specified in the template but do not exist in the resource group
does not reprovision resources that exist in the resource group in the same condition defined in the template
reprovisions existing resources that have updated settings in the template
To choose Incremental update or Complete update it depends on if you have resources that are in use. If devops requirement is to always have resources in sync with what is defined in the json template then Complete Update mode should be used. The biggest benefit of using templates and source code for deploying resources is to prevent configuration drift and it is beneficial to use Complete Update mode.
As for specifying the parameters if you specify in parameters file then you don't have to specify them again.
A new template can be deployed in incremental mode which would add new resources to the existing resource group. Define only the new resources in the template, existing resources would not be altered.
From powershell use the following cmdlet
New-AzureRmResourceGroupDeployment -ResourceGroupName "YourResourceGroupName" -TemplateFile "path\to\template.json" -Mode Incremental -Force
My rule of thumb is for things that I want to tear down, or for things I want to replicate across Subscriptions, I use ARM templates.
For example we want things in test environments, I just ARM it up, build on the scripts as developers request things ("Hey I need a cache", "Oh by the way I need to start using a Service Bus"), using incremental mode we can just push it out to Dev, then as we migrate up to different environments you just deploy to a different Subscription in Azure, and it should have everything ready to go.
Also, we've started provisioning our own Cloud Load Test agents in a VMSS, a simple ARM template that's called by a build to scale up to x number of machines, then when done, we just trash the Resource Group. It's repeatable and reliable, sure you can script it up, but as TFS has a task to deploy these things (also with schedules)...
One of the beautiful things I've come across is Key Vault, when you ARM it up and poke all the values from your service busses, storage accounts/whatevers, you can simply get the connection strings/keys/whatevers and put them straight into the Key Vault, so you never need to worry about it, and if you want to regenerate anything (say a dev wants to change the name of a cache or whatever, or accidentally posted the key to GitHub), just redeploy (often I'll just trash the whole Resource Group) and it updates the vault for you.
What I am interested in is reading the output parameters of another deployment in a different resource group.
My ARM templates are something like:
platform.json - sets up the DNS, virtual networks and security
storage.json - sets up databases and other stores
app.json - sets up the web app/api
Each is deployed in different resource groups as they have different life cycles. However, when I deploy the app.json I want to pull in the outputs of the latest platform and storage deployments and use them to configure the app.
Linked templates are not a solution as the linked templates end up deployed in the same resource group as the app which defeats the purpose of segregating your resources in resource groups.
Is there any way I can read the output parameters of a deployment from a different resource group? if not, is Azure planning to support it?
I know there is a way to get the resources by id, using the resourceId function, and look at their properties but I am trying to avoid doing that to not get into a resource reference spagetti.
I know this is an old question but for others who come along, as of 03/12/18 you can definitely do this.
You need to ensure your output is formatted as per the Microsoft documentation for output variables which broadly has the format
"outputs": {
"resourceID": {
"type": "string",
"value": "[resourceId('Microsoft.Network/publicIPAddresses', parameters('publicIPAddresses_name'))]"
}
}
You can then use these outputs in your templates by referencing the deployment using a resource reference which has the format
reference(resourceName or resourceIdentifier, [apiVersion], ['Full'])
Note that you will need to provide the api version, as the deployment may use a different api version to the one your parent template uses.
Your reference would then look something like the following
{
"comments": "This would have an output named myOutput you want to use",
"apiVersion": "2017-05-10",
"type": "Microsoft.Resources/deployments",
"name": "my-deployment",
"resourceGroup": "...",
"properties": {
"mode": "Incremental",
"templateLink": {
"uri": "...",
"contentVersion": "1.0.0.0"
},
"parameters": { }
},
{
"comments": "This makes use of myOutput from my-deployment",
"apiVersion": "2017-05-10",
"type": "Microsoft.Resources/deployments",
"name": "my-dependent-deployment",
"properties": {
"mode": "Incremental",
"templateLink": {
"uri": "...",
"contentVersion": "1.0.0.0"
},
"parameters": {
"myValueFromAnotherDeployment": { "value": "[reference('my-deployment', '2017-05-10').outputs.myOutput.value]" }
}
}
}
Note the slightly awkward "repackaging" of the value, where we use myOutput.value as the input to the dependent deployment, and put that in an object with the key "value": "....". This is because ARM parameters must have a 'value' property in order to be valid.
You'll get invalid template errors if you try to use the output directly (because output variables have a 'type' and this is not an allowed key in a parameter). That's why you need to get the value property and then put it back into the value in downstream templates.
How are you doing your deployments? In PowerShell you can do something like:
(Get-AzureResourceGroupDeployment NameOfResourceGroup).Outputs.NameOfOuputProperty.Value
And that will give you the output of the most recent deployment. You can also throw the entire deployment object into a var and have at it that way.
$d = Get-AzureResourceGroupDeployment NameOfResourceGroup
Which would be faster if you needed a many output properties.
That help?
Update for AzureRM cmdlet
The syntax is largely the same:
(Get-AzureRmResourceGroupDeployment -ResourceGroupName NameOfResourceGroup -Name NameOfDeployment).Outputs.NameOfOutputProperty.value
If you have multiple deployments you can use:
Get-AzureRmResourceGroupDeployment -ResourceGroupName NameOfResourceGroup
To see them all and see what the names are...
Are you using Azure DevOps Release Pipelines? You could just set the output to be created as variables so you can re-use them in the same or a different stage.
We use these extensions on our projects
ARM Outputs
https://github.com/keesschollaart81/vsts-arm-outputs
VSTS replacetokens
https://github.com/qetza/vsts-replacetokens-task