How to create shared image based off existing VM in Azure? - azure

I have an existing Virtual Machine running in Azure that has customised software installed. I want to use Terraform to create an image of this virtual machine and store it in an image gallery. The problem is, I dont understand how Terraform uniquely identifies the virtual machine in question.
Currently, I have the following:
// Get VM I want to create an image for (how can I use this as the image reference?)
data "azurerm_virtual_machine" "example" {
name = "example"
resource_group_name = "rg-example"
}
resource "azurerm_shared_image_gallery" "example" {
name = "example_image_gallery"
resource_group_name = azurerm_resource_group.example.name
location = azurerm_resource_group.example.location
description = "Shared images and things."
}
resource "azurerm_shared_image" "example" {
name = "my-image"
gallery_name = azurerm_shared_image_gallery.example.name
resource_group_name = "rg-example"
location = "australiacentral"
os_type = "Linux"
identifier {
publisher = "teradata"
offer = "vantage-teradata-viewpoint"
sku = "teradata-viewpoint-single-system-hourly-new"
}
specialized = true
}
As far as I can tell, Terraform can only create the image based on the identifier block. But this does not uniquely identify my virtual machine. Am I missing something obvious?
My goal is to perform the "Capture" operation that is available via the Azure Portal via Terraform. How do I specify my source VM?

Through additional research, I found I needed an azurerm_shared_image_version resource. Here, I was able to reference my existing Virtual Machine via managed_image_id:
// Get clienttools VM information
data "azurerm_virtual_machine" "example" {
name = "test-virtual-machine"
resource_group_name = "rg-example"
}
resource "azurerm_shared_image_gallery" "example" {
name = "myGallery"
resource_group_name = "rg-example"
location = "australiacentral"
description = "Shared images and things."
}
resource "azurerm_shared_image" "example" {
name = "my-image"
gallery_name = azurerm_shared_image_gallery.example.name
resource_group_name = "rg-example"
location = "australiacentral"
os_type = "Windows"
identifier {
publisher = "MicrosoftWindowsServer"
offer = "WindowsServer"
sku = "2019-datacenter-gensecond"
}
// Set this as it defaults to V1
hyper_v_generation = "V2"
specialized = true
}
resource "azurerm_shared_image_version" "example" {
name = "0.0.1"
gallery_name = azurerm_shared_image_gallery.example.name
image_name = azurerm_shared_image.example.name
resource_group_name = "rg-example"
location = "australiacentral"
managed_image_id = data.azurerm_virtual_machine.example.id
target_region {
name = "australiacentral"
regional_replica_count = 1
storage_account_type = "Standard_LRS"
}
}

Related

Is version mandatory while creating an Azure VM using terraform?

So I have been working with terraform since last 3 weeks and have been trying to use it to create self hosted GitHub Actions runners in our Azure account.
We have a shared windows VM image in Azure Compute Gallery that I'm planning to use as base image for the GA runner. I have noticed that these shared windows VM images do not generally have any versions attached to them they just have a publisher, offer and SKU attached.
I also verified by creating a new image from a VM to check if somebody missed attaching the version to the VM, but no shared images do not really have a version attached.
Yeah they do have versions but it is not attached as it is for Microsoft Platform Images.
Example of a shared image:
Now I found that in terraform, runners can be created by using both: azurerm_windows_virtual_machine and azurerm_virtual_machine resources.
I used both of them to test the runner creation, below are the terraform code used:
data "azurerm_shared_image" "win19_gold_image" {
provider = azurerm.gi
name = "Windows-2019_base"
gallery_name = data.azurerm_shared_image_gallery.cap_win_gold_image_gallery.name
resource_group_name = "gi-rg"
}
resource "azurerm_virtual_machine" "win_runners_gold_image_based" {
provider = azurerm.og
name = "ga-win-gold-1"
location = "East US"
count = "1" # if I need to increase the number of VMs.
resource_group_name = data.azurerm_resource_group.dts_rg.name
network_interface_ids = [azurerm_network_interface.azure_win_runner_gold_nic[count.index].id,]
vm_size = "Standard_D4ads_v5"
delete_os_disk_on_termination = true
delete_data_disks_on_termination = true
storage_image_reference {
publisher = data.azurerm_shared_image.win19_gold_image.identifier[0].publisher
offer = data.azurerm_shared_image.win19_gold_image.identifier[0].offer
sku = data.azurerm_shared_image.win19_gold_image.identifier[0].sku
# Here I get the error: Error: compute.VirtualMachinesClient#CreateOrUpdate: Failure sending request: StatusCode=400 -- Original Error: Code="InvalidParameter" Message="The value of parameter imageReference.version is invalid." Target="imageReference.version"
}
storage_os_disk {
name = "ga-win-gold-os-disk-1"
caching = "None"
create_option = "FromImage"
managed_disk_type = "StandardSSD_LRS"
}
os_profile {
computer_name = "ga-win-gold-1"
admin_username = "svc"
admin_password = var.WINDOWS_ADMIN_PASS
}
os_profile_windows_config {
enable_automatic_upgrades = true
provision_vm_agent = true
}
storage_data_disk {
name = "ga-win-gold-data-disk-1"
caching = "None"
create_option = "Empty"
disk_size_gb = var.disk_size_gb
lun = 0
managed_disk_type = "StandardSSD_LRS"
}
}
OR
data "azurerm_shared_image" "win19_gold_image" {
provider = azurerm.gi
name = "Windows-2019_base"
gallery_name = data.azurerm_shared_image_gallery.cap_win_gold_image_gallery.name
resource_group_name = "gi-rg"
}
resource "azurerm_windows_virtual_machine" "azure_win_runner" {
provider = azurerm.og
name = "vm-github-actions-win-${count.index}"
resource_group_name = data.azurerm_resource_group.dts_rg.name
location = "East US"
size = var.windows-vm-size
count = "${var.number_of_win_az_instances}"
network_interface_ids = [
azurerm_network_interface.azure_win_runner_nic[count.index].id,
]
computer_name = "vm-ga-win-${count.index}"
admin_username = var.windows-admin-username
admin_password = var.WINDOWS_ADMIN_PASS
os_disk {
name = "vm-github-actions-win-${count.index}-os-disk"
caching = "None"
storage_account_type = "StandardSSD_LRS"
}
source_image_reference {
publisher = data.azurerm_shared_image.win19_gold_image.identifier[0].publisher
offer = data.azurerm_shared_image.win19_gold_image.identifier[0].offer
sku = data.azurerm_shared_image.win19_gold_image.identifier[0].sku
version = data.azurerm_shared_image.win19_gold_image.identifier[0].version # says this object does not have a version attached to it.
# or version = "latest" or any other correct version string will throw error at time of apply that such a version does not exist.
}
enable_automatic_updates = true
provision_vm_agent = true
}
If I'm using azurerm_virtual_machine then if I ignore the version in storage_image_reference I receive the error:
Error: compute.VirtualMachinesClient#CreateOrUpdate: Failure sending request: StatusCode=400 -- Original Error: Code="InvalidParameter" Message="The value of parameter imageReference.version is invalid." Target="imageReference.version"
And if I add the version then I receive the error
Error: Unsupported attribute.
This object does not have an attribute named "version".
When using azurerm_windows_virtual_machine if I remove the version argument terraform complains that version is required and when provided a sting such as 1.0.0 or latest, while applying(terraform apply) it would complain that such a version does not exist.
And if I pull the version from data.azurerm_shared_image.cap_win19_gold_image it would complain that this object does not have a version.
I am confused as to how to use shared images for VM creation using terraform if version is mandatory yet if version is not available for azure shared images. Please advise on what am I missing?
Any help would be appreciated.
Thanks,
Sekhar
It seems to get a version of the image you need to use another resource [1] and another data source [2]:
data "azurerm_image" "win19_gold_image" {
name = "Windows-2019_base"
resource_group_name = "gi-rg"
}
resource "azurerm_shared_image_version" "win19_gold_image" {
name = "0.0.1"
gallery_name = data.azurerm_shared_image.win19_gold_image.gallery_name
image_name = data.azurerm_shared_image.win19_gold_image.name
resource_group_name = data.azurerm_shared_image.win19_gold_image.resource_group_name
location = data.azurerm_shared_image.win19_gold_image.location
managed_image_id = data.azurerm_image.win19_gold_image.id
}
And then in the source_image_reference block in the azurerm_windows_virtual_machine resource:
source_image_reference {
publisher = data.azurerm_shared_image.win19_gold_image.identifier[0].publisher
offer = data.azurerm_shared_image.win19_gold_image.identifier[0].offer
sku = data.azurerm_shared_image.win19_gold_image.identifier[0].sku
version = azurerm_shared_image_version.win19_gold_image.name
}
As it seems the name argument is actually the version of the image [3]:
name - (Required) The version number for this Image Version, such as 1.0.0. Changing this forces a new resource to be created.
[1] https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/data-sources/shared_image_version
[2] https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/data-sources/image
[3] https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/shared_image_version#name
Hi All who come across this question,
I found the solution to my issue. All I had to do was define a azurerm_shared_image_version data and then use source_image_id in azurerm_windows_virtual_machine in place of source_image_reference{} block.
Below is what I did:
data "azurerm_shared_image_gallery" "win_gold_image_gallery" {
provider = azurerm.gi
name = "golden_image_gallery"
resource_group_name = "gi-rg"
}
data "azurerm_shared_image" "win19_gold_image" {
provider = azurerm.gi
name = "Windows-2019_base"
gallery_name = data.azurerm_shared_image_gallery.win_gold_image_gallery.name
resource_group_name = data.azurerm_shared_image_gallery.win_gold_image_gallery.resource_group_name
}
data "azurerm_shared_image_version" "win19_gold_image_version" {
provider = azurerm.gi
name = "latest" # "recent" is also a tag to use the most recent image version
image_name = data.azurerm_shared_image.win19_gold_image.name
gallery_name = data.azurerm_shared_image.win19_gold_image.gallery_name
resource_group_name = data.azurerm_shared_image.win19_gold_image.resource_group_name
}
resource "azurerm_windows_virtual_machine" "azure_win_gi_runner" {
provider = azurerm.dep
name = "vm-github-actions-win-gi-${count.index}"
resource_group_name = data.azurerm_resource_group.dts_rg.name
location = "East US"
size = var.windows-vm-size
count = "${var.number_of_win_gi_az_instances}"
network_interface_ids = [
azurerm_network_interface.azure_win_gi_runner_nic[count.index].id,
]
computer_name = "ga-win-gi-${count.index}"
admin_username = var.windows-admin-username
admin_password = var.WINDOWS_ADMIN_PASS
os_disk {
name = "vm-github-actions-win-gi-${count.index}-os-disk"
caching = "None"
storage_account_type = "StandardSSD_LRS"
}
source_image_id = data.azurerm_shared_image_version.win19_gold_image_version.id
# This is the thing I was missing.
enable_automatic_updates = true
provision_vm_agent = true
tags = {
whichVM = var.gh_windows_runner
environment = var.environment
}
}

Ho to provision Terraform Azure Linux VM - You have not accepted the legal terms on this subscription

I am using terraform with azure to provision an ubuntu virtual machine and I am getting the below error:
creating Linux Virtual Machine: (Name "test-bastion" / Resource Group "ssi-test"): compute.VirtualMachinesClient#CreateOrUpdate: Failure sending request: StatusCode=400 -- Original Error: Code="ResourcePurchaseValidationFailed" Message="User failed validation to purchase resources. Error message: 'You have not accepted the legal terms on this subscription: 'xxxxx-xxxxx-xxxxx-xxxx' for this plan.
I can spin up VM's through azure portal but not with terraform.
Here's my terraform module
resource "azurerm_linux_virtual_machine" "linux_virtual_machine" {
name = join("-", [var.environment, "bastion"])
resource_group_name = var.resource_group_name
location = var.location
size = var.bastion_size
admin_username = var.bastion_admin_username
computer_name = join("-", [var.project, var.environment, "bastion"])
custom_data = filebase64(var.bastion_custom_data_path)
network_interface_ids = [
azurerm_network_interface.bastion_nic.id
]
admin_ssh_key {
username = var.bastion_admin_username
public_key = file(var.bastion_public_key_path)
}
source_image_reference {
publisher = var.bastion_publisher
offer = var.bastion_offer
sku = var.bastion_sku
version = var.bastion_version
}
plan {
name = var.bastion_sku
publisher = var.bastion_publisher
product = var.bastion_offer
}
os_disk {
name = join("-", [var.project, var.environment, "bastion-os-disk"])
storage_account_type = "Standard_LRS"
caching = "ReadWrite"
disk_size_gb = var.bastion_os_disk_size_gb
}
}
# Create network interface
resource "azurerm_network_interface" "bastion_nic" {
name = join("-", [var.project, var.environment, "bastion-nic"])
location = var.location
resource_group_name = var.resource_group_name
depends_on = [azurerm_public_ip.bastion_public_ip]
ip_configuration {
name = join("-", [var.project, var.environment, "bastion-nic-conf"])
subnet_id = var.bastion_subnet_id
private_ip_address_allocation = "Dynamic"
public_ip_address_id = azurerm_public_ip.bastion_public_ip.id
}
tags = var.default_tags
}
and here are the variable values (some are removed)
bastion_admin_username = "ubuntu"
bastion_os_disk_size_gb = "60"
bastion_public_key_path = "./data/keys/bastion.pub"
bastion_size = "Standard_B2s"
bastion_publisher = "canonical"
bastion_offer = "0001-com-ubuntu-server-focal"
bastion_sku = "20_04-lts-gen2"
bastion_version = "latest"
bastion_custom_data_path = "./data/scripts/bastion.sh"
Can someone help me?
Plan block is mostly for BYOS images like RedHat, Arista & Palo Alto. Below flavor doesn't need any plan as this can be used without accepting marketplace terms first before using them via automation.
> az vm image list-skus -l westeurope -p canonical -f 0001-com-ubuntu-server-focal
{
"extendedLocation": null,
"id": "/Subscriptions/b500a058-6396-45db-a15d-3f31913e84a5/Providers/Microsoft.Compute/Locations/westeurope/Publishers/canonical/ArtifactTypes/VMImage/Offers/0001-com-ubuntu-server-focal/Skus/20_04-lts-gen2",
"location": "westeurope",
"name": "20_04-lts-gen2",
"properties": {
"automaticOSUpgradeProperties": {
"automaticOSUpgradeSupported": false
}
},
"tags": null
}
If you remove below plan block from azurerm_linux_virtual_machine resource, it should work for the image flavor you picked.
plan {
name = var.bastion_sku
publisher = var.bastion_publisher
product = var.bastion_offer
}
The reason why it's working via portal because ARM template doesn't add plan block there. You can download and verify ARM template before creating VM on portal if you want.
Accept the agreement first, with this resource: https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/marketplace_agreement

Deploy Azure VM based on customize Image available on Shared Image Gallery

I have first created a simple standard VM in Azure which I have customized for data science purpose with different software on it. Then I have created an image from this VM so that I can deploy new VM based on the customized image faster with same configuration. I have saved the image under Azure Shared Image Gallery.
Is it any way to deploy this customized image from a Terraform script into a new Resource Group? I know how to deploy a normal standard VM from Terraform but couldn't find out how to deploy it based on a customized image saved in the Shared Image gallery.
To deploy a customized image from Azure Shared Image Gallery with terraform. You could use Data Source: azurerm_shared_image and azurerm_windows_virtual_machine or azurerm_linux_virtual_machine to manage it with specify the source_image_id. Please note that the newly created VM should be in the same region as the shared image before you deploy it. If not, you could replicate this image to your desired region, read https://learn.microsoft.com/en-us/azure/virtual-machines/shared-image-galleries#replication
For example, deploy a Windows VM from generalized shared image:
provider "azurerm" {
features {}
}
data "azurerm_shared_image" "example" {
name = "my-image"
gallery_name = "my-image-gallery"
resource_group_name = "example-resources-imageRG"
}
resource "azurerm_resource_group" "example" {
name = "example-resources"
location = "xxxxximageregion"
}
resource "azurerm_virtual_network" "example" {
name = "example-network"
address_space = ["10.0.0.0/16"]
location = azurerm_resource_group.example.location
resource_group_name = azurerm_resource_group.example.name
}
resource "azurerm_subnet" "example" {
name = "internal"
resource_group_name = azurerm_resource_group.example.name
virtual_network_name = azurerm_virtual_network.example.name
address_prefixes = ["10.0.2.0/24"]
}
resource "azurerm_network_interface" "example" {
name = "example-nic"
location = azurerm_resource_group.example.location
resource_group_name = azurerm_resource_group.example.name
ip_configuration {
name = "internal"
subnet_id = azurerm_subnet.example.id
private_ip_address_allocation = "Dynamic"
}
}
resource "azurerm_windows_virtual_machine" "example" {
name = "example-machine"
resource_group_name = azurerm_resource_group.example.name
location = azurerm_resource_group.example.location
size = "Standard_F2"
admin_username = "adminuser"
admin_password = "P#$$w0rd1234!"
network_interface_ids = [
azurerm_network_interface.example.id,
]
os_disk {
caching = "ReadWrite"
storage_account_type = "Standard_LRS"
}
source_image_id = data.azurerm_shared_image.example.id
}
Result

azure function via terraform: how to connect to service bus

I am stuck when trying to deploy an Azure function via Azure DevOps pipelines and Terraform.
Running terraform apply works fine and the Service Bus looks good and works. In the Azure portal the function seems to be running, but it complains that it can not find the ServiceBusConnection.
I defined it via the following Terraform declaration:
resource "azurerm_resource_group" "rg" {
name = "rg-sb-westeurope"
location = "westeurope"
}
resource "azurerm_servicebus_namespace" "sb" {
name = "ns-sb"
location = azurerm_resource_group.rg.location
resource_group_name = azurerm_resource_group.rg.name
sku = "Standard"
}
resource "azurerm_servicebus_queue" "sbq" {
name = "servicebusqueue"
resource_group_name = azurerm_resource_group.rg.name
namespace_name = azurerm_servicebus_namespace.sb.name
enable_partitioning = true
}
resource "azurerm_servicebus_namespace_authorization_rule" "sb-ar" {
name = "servicebus_auth_rule"
namespace_name = azurerm_servicebus_namespace.sb.name
resource_group_name = azurerm_resource_group.rg.name
listen = false
send = true
manage = false
}
In the function app i declare:
resource "azurerm_function_app" "fa" {
name = "function-app"
location = azurerm_resource_group.rg.location
resource_group_name = azurerm_resource_group.rg.name
app_service_plan_id = azurerm_app_service_plan.asp.id
storage_account_name = azurerm_storage_account.sa.name
storage_account_access_key = azurerm_storage_account.sa.primary_access_key
app_settings = {
ServiceBusConnection = azurerm_servicebus_namespace_authorization_rule.sb-ar.name
}
}
This tf. will not work out of the box as i have not copied here the full declaration.
I think I am setting the connection environment vars wrong but have no idea on how to do it correctly.
EDIT
With the hint from #Heye I got it working. This is the correct snipped replacing the name with primary_connection_string.
resource "azurerm_function_app" "fa" {
name = "function-app"
location = azurerm_resource_group.rg.location
resource_group_name = azurerm_resource_group.rg.name
app_service_plan_id = azurerm_app_service_plan.asp.id
storage_account_name = azurerm_storage_account.sa.name
storage_account_access_key = azurerm_storage_account.sa.primary_access_key
app_settings = {
ServiceBusConnection = azurerm_servicebus_namespace_authorization_rule.sb-ar.primary_connection_string
}
}
You are setting the ServiceBusConnection value to the name of the authorization rule. However, you probably want to set it to the primary_connection_string, as that contains the key along with all the information needed to connect to the Service Bus.

azure terraform attaching azure file share to windows machine

Problem statement
I am in the process to create an Azure VM cluster of windows os. till now I can create an Azure file share. and azure windows cluster. I want to attach file share created to each VM in my cluster. unable to find reference how to add same on windows VM.
code for this
resource "azurerm_storage_account" "main" {
name = "stor${var.environment}${var.cost_centre}${var.project}"
location = "${azurerm_resource_group.main.location}"
resource_group_name = "${azurerm_resource_group.main.name}"
account_tier = "${var.storage_account_tier}"
account_replication_type = "${var.storage_replication_type}"
}
resource "azurerm_storage_share" "main" {
name = "storageshare${var.environment}${var.cost_centre}${var.project}"
resource_group_name = "${azurerm_resource_group.main.name}"
storage_account_name = "${azurerm_storage_account.main.name}"
quota = "${var.storage_share_quota}"
}
resource "azurerm_virtual_machine" "vm" {
name = "vm-${var.location_id}-${var.environment}-${var.cost_centre}-${var.project}-${var.seq_id}-${count.index}"
location = "${azurerm_resource_group.main.location}"
resource_group_name = "${azurerm_resource_group.main.name}"
availability_set_id = "${azurerm_availability_set.main.id}"
vm_size = "${var.vm_size}"
network_interface_ids = ["${element(azurerm_network_interface.main.*.id, count.index)}"]
count = "${var.vm_count}"
storage_image_reference {
publisher = "${var.image_publisher}"
offer = "${var.image_offer}"
sku = "${var.image_sku}"
version = "${var.image_version}"
}
storage_os_disk {
name = "osdisk${count.index}"
create_option = "FromImage"
}
os_profile {
computer_name = "${var.vm_name}-${count.index}"
admin_username = "${var.admin_username}"
admin_password = "${var.admin_password}"
}
os_profile_windows_config {}
depends_on = ["azurerm_network_interface.main"]
}
Azure doesnt offer anything like that, so uou cannot do that natively, you need to create a script and run that script on the vm use script extension\dsc extension or if terraform supports that - with terraform.
https://learn.microsoft.com/en-us/azure/virtual-machines/extensions/custom-script-linux

Resources