I'm running trying to run a bash script on an Azure Linux VM scaleset using custom script extensions, I have the script uploaded into an Azure Storage account already. The bash script is meant to install ngix, on the VM Scaleset. The script runs without any errors, however if I log into any of the VMScaleset instances to validate I don't see NGIX running.
Bash script here
#!/bin/bash
apt-get update
apt-get install -y nginx
Terraform file here
data "azurerm_subnet" "refdata" {
name = var.subnetName1
virtual_network_name = var.vnetName
resource_group_name = var.resourceGroupName
}
resource "azurerm_windows_virtual_machine_scale_set" "res-vmscaleset" {
name = var.vmScaleSetName
resource_group_name = azurerm_resource_group.DevRG.name
location = azurerm_resource_group.DevRG.location
sku = "Standard_F2"
instances = 1
admin_password = "xxxxxx"
admin_username = "adminuser"
source_image_reference {
publisher = "MicrosoftWindowsServer"
offer = "WindowsServer"
sku = "2016-Datacenter-Server-Core"
version = "latest"
}
os_disk {
storage_account_type = "Standard_LRS"
caching = "ReadWrite"
}
network_interface {
name = "vmscaleset-nic"
primary = true
ip_configuration {
name = "internal"
primary = true
subnet_id=data.azurerm_subnet.test.id
}
}
}
resource "azurerm_linux_virtual_machine_scale_set" "res-linuxscale" {
name = "linuxvmss"
resource_group_name = azurerm_resource_group.DevRG.name
location = azurerm_resource_group.DevRG.location
sku = "Standard_F2"
instances = 2
admin_password = "Password1234!"
disable_password_authentication = false
admin_username = "adminuser"
source_image_reference {
publisher = "Canonical"
offer = "UbuntuServer"
sku = "16.04-LTS"
version = "latest"
}
os_disk {
storage_account_type = "Standard_LRS"
caching = "ReadWrite"
}
network_interface {
name = "lvmscaleset-nic"
primary = true
ip_configuration {
name = "internal"
primary = true
subnet_id=data.azurerm_subnet.test.id
}
}
}
resource "azurerm_virtual_machine_scale_set_extension" "res-extension" {
name = "example"
virtual_machine_scale_set_id = azurerm_linux_virtual_machine_scale_set.res-linuxscale.id
publisher = "Microsoft.OSTCExtensions"
type = "CustomScriptForLinux"
type_handler_version = "1.0"
settings = <<SETTINGS
{
"fileUris": ["https://xxxxxxxxxxx.blob.core.windows.net/shellscript11/post-deploy.sh"],
"commandToExecute": "sh post-deploy.sh"
}
SETTINGS
}
Reference to this document, you can use the publisher and type for your custom script like this.
resource "azurerm_virtual_machine_scale_set_extension" "res-extension" {
name = "nnn-extension"
virtual_machine_scale_set_id = azurerm_linux_virtual_machine_scale_set.example.id
publisher = "Microsoft.Azure.Extensions"
type = "CustomScript"
type_handler_version = "2.0"
settings = jsonencode({
"fileUris" = ["https://xxxx.blob.core.windows.net/shscripts/aptupdate.sh"],
"commandToExecute" = "sh aptupdate.sh"
}
)
}
After applying the above configurations, you could upgrade each vmss instance, then the Nginx will be running.
Result
Related
I'm running azurerm_mssql_virtual_machine to build a SQL Server virtual machine from a custom imag. (Image configured with SQL Server 2016 prepare image).
This is the code that I am running:
resource "azurerm_mssql_virtual_machine" "mssql_vm" {
provider = azurerm.spoke-subscription
virtual_machine_id = azurerm_windows_virtual_machine.sql_server.id
sql_license_type = "PAYG"
sql_connectivity_port = "49535"
sql_connectivity_update_username = var.sql_login
sql_connectivity_update_password = var.sql_password
sql_instance {
collation = "Latin1_General_CI_AS"
}
assessment {
enabled = true
run_immediately = true
}
storage_configuration {
disk_type = "${var.disk_type}"
storage_workload_type = "OLTP"
data_settings {
default_file_path = "F:\\DATA"
luns = [1]
}
log_settings {
default_file_path = "G:\\LOGS"
luns = [2]
}
temp_db_settings {
default_file_path = "K:\\TEMPDB"
luns = [3]
}
}
lifecycle {
ignore_changes = [
tags,
#assessment[0].schedule
]
}
tags = {
"application owner" = var.application_owner_tag
"environment" = var.environment_tag
"department" = var.department_tag
"technicalcontact" = var.technicalcontact_tag
"application" = var.application_tag
"service" = "SQL server"
}
}
I get this error:
performing CreateOrUpdate: sqlvirtualmachines.SqlVirtualMachinesClient#CreateOrUpdate: Failure sending request: StatusCode=0 --
Original Error: Code="CRPNotAllowedOperation" Message="Operation cannot be completed due to the following error: VM Extension with publisher 'Microsoft.SqlServer.Management' and type 'SqlIaaSAgent' does not support setting enableAutomaticUpgrade property to true on this subscription.
Steps I've taken to try and resolve:
Re-register SQL Server virtual machines to the Azure subscription
Turned off automatic upgrade on azurerm_windows_virtual_machine
I tried to reproduce the same in my environment:
Code:
resource "azurerm_mssql_virtual_machine" "example" {
virtual_machine_id = azurerm_windows_virtual_machine.example.id
sql_license_type = "PAYG"
r_services_enabled = true
sql_connectivity_port = 1433
sql_connectivity_type = "PRIVATE"
sql_connectivity_update_password = "xxx"
sql_connectivity_update_username = "sqllogin"
auto_patching {
day_of_week = "Sunday"
maintenance_window_duration_in_minutes = 60
maintenance_window_starting_hour = 2
}
}
resource "azurerm_virtual_network" "example" {
name = "kavexample-network"
address_space = ["10.0.0.0/16"]
location = data.azurerm_resource_group.example.location
resource_group_name = data.azurerm_resource_group.example.name
}
resource "azurerm_subnet" "example" {
name = "internal"
resource_group_name = data.azurerm_resource_group.example.name
virtual_network_name = azurerm_virtual_network.example.name
address_prefixes = ["10.0.2.0/24"]
}
resource "azurerm_network_interface" "example" {
name = "kavya-example-nic"
location = data.azurerm_resource_group.example.location
resource_group_name = data.azurerm_resource_group.example.name
ip_configuration {
name = "internal"
subnet_id = azurerm_subnet.example.id
private_ip_address_allocation = "Dynamic"
}
}
resource "azurerm_windows_virtual_machine" "example" {
name = "kavyaexamplemc"
resource_group_name = data.azurerm_resource_group.example.name
location = data.azurerm_resource_group.example.location
size = "Standard_F2"
admin_username = "xxx"
admin_password = "xx"
enable_automatic_updates = true
patch_mode = "Manual"
hotpatching_enabled = true
network_interface_ids = [
azurerm_network_interface.example.id,
]
os_disk {
caching = "ReadWrite"
storage_account_type = "Standard_LRS"
}
source_image_reference {
publisher = "MicrosoftWindowsServer"
offer = "WindowsServer"
sku = "2016-Datacenter"
version = "latest"
}
}
Received same error:
│ Error: waiting for creation of Sql Virtual Machine (Sql Virtual Machine Name "kavyaexamplemc" / Resource Group "v-sakavya-Mindtree"): Code="CRPNotAllowedOperation" Message="Operation cannot be completed due to the following error: VM Extension with publisher 'Microsoft.SqlServer.Management' and type 'SqlIaaSAgent' does not support setting enableAutomaticUpgrade property to true on this subscription."
Even tried changing, but was still receving the same error again and again.
enable_automatic_updates = false
patch_mode = "Manual"
hotpatching_enabled = false
Try deleting the vm resource completely and create a new one with changed settings .
Try using below code:
I tried setting enable_automatic_upgrades = false , azurerm_virtual_machine has this property .Make use of that.
Also ,
Code:
resource "azurerm_virtual_network" "main" {
name = "kavyasarvnetwork"
address_space = ["10.0.0.0/16"]
location = data.azurerm_resource_group.example.location
resource_group_name = data.azurerm_resource_group.example.name
}
resource "azurerm_subnet" "internal" {
name = "internal"
resource_group_name = data.azurerm_resource_group.example.name
virtual_network_name = azurerm_virtual_network.main.name
address_prefixes = ["10.0.2.0/24"]
}
resource "azurerm_network_interface" "main" {
name = "kavyasarnic"
location = data.azurerm_resource_group.example.location
resource_group_name = data.azurerm_resource_group.example.name
ip_configuration {
name = "testconfiguration1"
subnet_id = azurerm_subnet.internal.id
private_ip_address_allocation = "Dynamic"
}
}
resource "azurerm_virtual_machine" "example" {
name = "kavyasarvm"
location = data.azurerm_resource_group.example.location
resource_group_name = data.azurerm_resource_group.example.name
network_interface_ids = [azurerm_network_interface.main.id]
vm_size = "Standard_DS1_v2"
storage_os_disk {
name = "kavyasar-OSDisk"
caching = "ReadOnly"
create_option = "FromImage"
managed_disk_type = "Premium_LRS"
os_type = "Windows"
}
storage_image_reference {
publisher = "MicrosoftSQLServer"
offer = "SQL2017-WS2016"
sku = "SQLDEV"
version = "latest"
}
os_profile {
computer_name = "hostname"
admin_username = "testadmin"
admin_password = "Password1234!"
}
os_profile_windows_config {
timezone = "Pacific Standard Time"
provision_vm_agent = true
enable_automatic_upgrades = false
}
tags = {
environment = "staging"
}
}
resource "azurerm_mssql_virtual_machine" "example" {
virtual_machine_id = azurerm_virtual_machine.example.id
sql_license_type = "PAYG"
r_services_enabled = true
sql_connectivity_port = 1433
sql_connectivity_type = "PRIVATE"
sql_connectivity_update_password = "Password1234!"
sql_connectivity_update_username = "sqllogin"
}
This seems to be the cause due to limitations: What is the SQL Server IaaS Agent extension? (Windows) - SQL Server on Azure VMs | Microsoft Learn
The SQL IaaS Agent extension only supports:
SQL Server VMs deployed through the Azure Resource Manager. SQL Server
VMs deployed through the classic model are not supported.
SQL Server VMs deployed to the public or Azure Government cloud.
Deployments to other private or government clouds are not supported.
Reference : azurerm_mssql_virtual_machine | Resources | hashicorp/azurerm | Terraform Registry
When deploying a custom script extension for a VM in Azure, it times out after 15 minutes. The timeout block is set to 2hrs. I cannot figure out why it keeps timing out. Could anyone point me in the right direction please? Thanks.
Resource to deploy (https://i.stack.imgur.com/lIfKj.png)
Error (https://i.stack.imgur.com/GFYRL.png)
In Azure, each resource will take a particular amount of time for provisioning. For Virtual Network Gateway's/ Virtual machines, timeout is up to 2 hours as mentioned in terraform timeouts.
Therefore, the timeout block we provide for any virtual machine has to be less than two hours (2h).
I tried creating a replica for azure vm extension resource by using below terraform code and it deployed successfully.
timeout block:
timeouts {
create = "1h30m"
delete = "20m"
}
azure_VM_extension:
resource "azurerm_virtual_machine_extension" "xxxxx" {
name = "xxxxname"
virtual_machine_id = azurerm_virtual_machine.example.id
publisher = "Microsoft.Azure.Extensions"
type = "CustomScript"
type_handler_version = "2.0"
settings = <<SETTINGS
{
"commandToExecute": "hostname && uptime"
}
SETTINGS
tags = {
environment = "Production"
}
timeouts {
create = "1h30m"
delete = "20m"
}
}
Created a virtual machine by adding required configurations under resource group.
main.tf:
terraform {
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = "=3.0.0"
}
}
}
provider "azurerm" {
features {}
}
resource "azurerm_resource_group" "xxxxxRG" {
name = "xxxxx-RG"
location = "xxxxxx"
}
resource "azurerm_virtual_network" "example" {
name = "xxxxx"
address_space = ["10.0.0.0/16"]
location = azurerm_resource_group.example.location
resource_group_name = azurerm_resource_group.example.name
}
resource "azurerm_subnet" "example" {
name = "xxxxx"
resource_group_name = azurerm_resource_group.example.name
virtual_network_name = azurerm_virtual_network.example.name
address_prefixes = ["10.0.2.0/24"]
}
resource "azurerm_network_interface" "example" {
name = "xxxxxx"
location = azurerm_resource_group.example.location
resource_group_name = azurerm_resource_group.example.name
ip_configuration {
name = "xxxxconfiguration"
subnet_id = azurerm_subnet.example.id
private_ip_address_allocation = "Dynamic"
}
}
resource "azurerm_storage_account" "example" {
name = "xxxxx"
resource_group_name = azurerm_resource_group.example.name
location = azurerm_resource_group.example.location
account_tier = "Standard"
account_replication_type = "LRS"
tags = {
environment = "staging"
}
}
resource "azurerm_storage_container" "example" {
name = "xxxxxx"
storage_account_name = azurerm_storage_account.example.name
container_access_type = "private"
}
resource "azurerm_virtual_machine" "example" {
name = "xxxxxxVM"
location = azurerm_resource_group.example.location
resource_group_name = azurerm_resource_group.example.name
network_interface_ids = [azurerm_network_interface.example.id]
vm_size = "Standard_F2"
storage_image_reference {
publisher = "Canonical"
offer = "UbuntuServer"
sku = "16.04-LTS"
version = "latest"
}
storage_os_disk {
name = "xxxxx"
vhd_uri = "${azurerm_storage_account.example.primary_blob_endpoint}${azurerm_storage_container.example.name}/myosdisk1.vhd"
caching = "ReadWrite"
create_option = "FromImage"
}
os_profile {
computer_name = "xxxxxname"
admin_username = "xxxx"
admin_password = "xxxxxx"
}
os_profile_linux_config {
disable_password_authentication = false
}
tags = {
environment = "staging"
}
}
resource "azurerm_virtual_machine_extension" "example" {
name = "hostname"
virtual_machine_id = azurerm_virtual_machine.example.id
publisher = "Microsoft.Azure.Extensions"
type = "CustomScript"
type_handler_version = "2.0"
settings = <<SETTINGS
{
"commandToExecute": "hostname && uptime"
}
SETTINGS
tags = {
environment = "Production"
}
timeouts {
create = "1h30m"
delete = "20m"
}
}
Executed:
terraform init:
terraform plan:
terraform apply:
Extension added successfully after deployment:
You can upgrade status if you want to use extensions.
I resolved the issue by changing the type_handler_version to 1.9.
Did someone managed to enable via terraforms Insights for a VM?
i'm able to create a VM, enable logging, but not enable insights..
i've seen this question: but don't find a clear answer..
How to enable azure vm application insights monitoring agent using terraform
Here is my full terraform script that i'm using for tests, i'm running it directly on the cloud shell from azure.
# Configure the Azure provider
provider "azurerm" {
# The "feature" block is required for AzureRM provider 2.x.
features {}
}
variable "prefix" {
default = "tfvmex"
}
resource "azurerm_resource_group" "main" {
name = "${var.prefix}-resources"
location = "West Europe"
}
resource "azurerm_virtual_network" "main" {
name = "${var.prefix}-network"
address_space = ["10.0.0.0/16"]
location = azurerm_resource_group.main.location
resource_group_name = azurerm_resource_group.main.name
}
resource "azurerm_subnet" "internal" {
name = "internal"
resource_group_name = azurerm_resource_group.main.name
virtual_network_name = azurerm_virtual_network.main.name
address_prefixes = ["10.0.2.0/24"]
}
resource "azurerm_network_interface" "main" {
name = "${var.prefix}-nic"
location = azurerm_resource_group.main.location
resource_group_name = azurerm_resource_group.main.name
ip_configuration {
name = "testconfiguration1"
subnet_id = azurerm_subnet.internal.id
private_ip_address_allocation = "Dynamic"
}
}
resource "azurerm_virtual_machine" "main" {
name = "${var.prefix}-vm"
location = azurerm_resource_group.main.location
resource_group_name = azurerm_resource_group.main.name
network_interface_ids = [azurerm_network_interface.main.id]
vm_size = "Standard_DS1_v2"
# Uncomment this line to delete the OS disk automatically when deleting the VM
# delete_os_disk_on_termination = true
# Uncomment this line to delete the data disks automatically when deleting the VM
# delete_data_disks_on_termination = true
storage_image_reference {
publisher = "Canonical"
offer = "UbuntuServer"
sku = "16.04-LTS"
version = "latest"
}
storage_os_disk {
name = "myosdisk1"
caching = "ReadWrite"
create_option = "FromImage"
managed_disk_type = "Standard_LRS"
}
os_profile {
computer_name = "hostname"
admin_username = "testadmin"
admin_password = "Password1234!"
}
os_profile_linux_config {
disable_password_authentication = false
}
tags = {
environment = "staging"
}
}
resource "azurerm_storage_account" "main" {
name = "omstesttest22"
resource_group_name = azurerm_resource_group.main.name
location = "westus"
account_tier = "Standard"
account_replication_type = "GRS"
tags = {
environment = "staging"
}
}
resource "azurerm_log_analytics_workspace" "law02" {
name = "${var.prefix}-logAnalytics"
location = azurerm_resource_group.main.location
resource_group_name = azurerm_resource_group.main.name
sku = "PerGB2018"
retention_in_days = 30
}
resource "azurerm_log_analytics_solution" "example" {
solution_name = "ContainerInsights"
location = azurerm_resource_group.main.location
resource_group_name = azurerm_resource_group.main.name
workspace_resource_id = azurerm_log_analytics_workspace.law02.id
workspace_name = azurerm_log_analytics_workspace.law02.name
plan {
publisher = "Microsoft"
product = "OMSGallery/ContainerInsights"
}
}
#===================================================================
# Set Monitoring and Log Analytics Workspace
#===================================================================
resource "azurerm_virtual_machine_extension" "oms_mma02" {
name = "test-OMSExtension"
virtual_machine_id = azurerm_virtual_machine.main.id
publisher = "Microsoft.EnterpriseCloud.Monitoring"
type = "OmsAgentForLinux"
type_handler_version = "1.12"
auto_upgrade_minor_version = true
settings = <<SETTINGS
{
"workspaceId" : "${azurerm_log_analytics_workspace.law02.workspace_id}"
}
SETTINGS
protected_settings = <<PROTECTED_SETTINGS
{
"workspaceKey" : "${azurerm_log_analytics_workspace.law02.primary_shared_key}"
}
PROTECTED_SETTINGS
}
Hope it was clear.
Thanks!
From the document, VM insights require the following two agents to be installed on each virtual machine to be monitored.
Log Analytics agent. Collects events and performance data from the virtual machine or virtual machine scale set and delivers it to the Log Analytics workspace. Deployment methods for the Log Analytics agent on Azure resources use the VM extension for Windows and Linux.
Dependency agent. Collects discovered data about processes running on the virtual machine and external process dependencies, which are used by the Map feature in VM insights. The Dependency agent relies on the Log Analytics agent to deliver its data to Azure Monitor. Deployment methods for the Dependency agent on Azure resources use the VM extension for Windows and Linux.
After my validation, you can add the DependencyAgent extension to your existing code.
resource "azurerm_virtual_machine_extension" "da" {
name = "DAExtension"
virtual_machine_id = azurerm_virtual_machine.main.id
publisher = "Microsoft.Azure.Monitoring.DependencyAgent"
type = "DependencyAgentLinux"
type_handler_version = "9.5"
auto_upgrade_minor_version = true
}
For more information, read Configure Log Analytics workspace for VM insights and Enable VM insights guest health (preview)
please use the product "OMSGallery/VMInsights" (instead of "OMSGallery/ContainerInsights")
resource "azurerm_log_analytics_solution" "..." {
solution_name = "..."
location = ...
resource_group_name = ...
workspace_resource_id = ...
workspace_name = ...
plan {
publisher = "Microsoft"
product = "OMSGallery/VMInsights"
}
}
To deploy it using Terraform:
Deploy a log analytics workspace and a VMInsights solution associated with the workspace.
resource "azurerm_log_analytics_workspace" "law" {
name = "LogAnalyticsWorkspace"
location = "Your location"
resource_group_name = "Your resource group"
sku = "PerGB2018"
retention_in_days = "your retention in days"
internet_ingestion_enabled= true
internet_query_enabled = false
tags = "Your tags"
}
resource "azurerm_log_analytics_solution" "vminsights" {
solution_name = "VMInsights"
location = "Your location"
resource_group_name = "Your resource group"
workspace_resource_id = azurerm_log_analytics_workspace.law.id
workspace_name = azurerm_log_analytics_workspace.law.name
tags = "Your tags"
plan {
publisher = "Microsoft"
product = "OMSGallery/VMInsights"
}
}
Deploy VM with as usual with OMSAgent and DependencyAgentWindows extensions:
resource "azurerm_windows_virtual_machine" "vm" {
......
......
}
OMS for Windows:
https://learn.microsoft.com/en-us/azure/virtual-machines/extensions/oms-windows
resource "azurerm_virtual_machine_extension" "omsext" {
name = "OMSExtension"
virtual_machine_id = azurerm_windows_virtual_machine.vm.id
publisher = "Microsoft.EnterpriseCloud.Monitoring"
type = "MicrosoftMonitoringAgent"
type_handler_version = "1.0"
auto_upgrade_minor_version = true
settings = <<SETTINGS
{
"workspaceId": "${azurerm_log_analytics_workspace.law.id}"
}
SETTINGS
protected_settings = <<PROTECTED_SETTINGS
{
"workspaceKey": "${azurerm_log_analytics_workspace.law.primary_shared_key}"
}
PROTECTED_SETTINGS
tags = "Your tags"
}
DA Agent for Windows:
https://learn.microsoft.com/en-us/azure/virtual-machines/extensions/agent-dependency-windows
resource "azurerm_virtual_machine_extension" "DAAgent" {
name = "DAAgentExtension"
virtual_machine_id = azurerm_windows_virtual_machine.vm.id
publisher = "Microsoft.Azure.Monitoring.DependencyAgent"
type = "DependencyAgentWindows"
type_handler_version = "9.10"
auto_upgrade_minor_version = true
tags = "Your tags"
}
Microsoft have changed the settings needed in the MicrosoftMonitoringAgent extensions, and the terraform specified by #Bill no longer works as of June 2022. The Terraform that worked for me was:
# Import the subscription and resource groups
data "azurerm_subscription" "current" {
}
data "azurerm_resource_group" "rg" {
name = "rg-name"
provider = azurerm
}
resource "random_password" "windowsvm-password" {
length = 24
special = false
}
# Define the VM itself
resource "azurerm_windows_virtual_machine" "windowsvm-c" {
name = "mywindowsvm"
computer_name = "mywindowsvm"
resource_group_name = data.azurerm_resource_group.rg.name
location = data.azurerm_resource_group.rg.location
size = "Standard_B2s"
admin_username = "adminlogin"
admin_password = random_password.windowsvm-password.result
identity { type = "SystemAssigned" }
network_interface_ids = [
azurerm_network_interface.windowsvm-c-nic.id,
]
os_disk {
caching = "ReadWrite"
storage_account_type = "Standard_LRS"
}
source_image_reference {
publisher = "MicrosoftWindowsServer"
offer = "WindowsServer"
sku = "2022-datacenter-azure-edition-core"
version = "latest"
}
patch_mode = "AutomaticByPlatform"
hotpatching_enabled = true
}
# Add logging and monitoring
resource "azurerm_log_analytics_workspace" "law" {
name = "vmloganalytics"
resource_group_name = data.azurerm_resource_group.rg-c.name
location = data.azurerm_resource_group.rg-c.location
sku = "PerGB2018"
retention_in_days = 365
internet_ingestion_enabled= true
internet_query_enabled = false
}
resource "azurerm_log_analytics_solution" "vminsights" {
solution_name = "vminsights"
resource_group_name = data.azurerm_resource_group.rg-c.name
location = data.azurerm_resource_group.rg-c.location
workspace_resource_id = azurerm_log_analytics_workspace.law.id
workspace_name = azurerm_log_analytics_workspace.law.name
plan {
publisher = "Microsoft"
product = "VMInsights"
}
}
# This extension is needed for other extensions
resource "azurerm_virtual_machine_extension" "daa-agent" {
name = "DependencyAgentWindows"
virtual_machine_id = azurerm_windows_virtual_machine.windowsvm-c.id
publisher = "Microsoft.Azure.Monitoring.DependencyAgent"
type = "DependencyAgentWindows"
type_handler_version = "9.10"
automatic_upgrade_enabled = true
auto_upgrade_minor_version = true
}
# Add logging and monitoring extensions
resource "azurerm_virtual_machine_extension" "monitor-agent" {
depends_on = [ azurerm_virtual_machine_extension.daa-agent ]
name = "AzureMonitorWindowsAgent"
virtual_machine_id = azurerm_windows_virtual_machine.windowsvm-c.id
publisher = "Microsoft.Azure.Monitor"
type = "AzureMonitorWindowsAgent"
type_handler_version = "1.5"
automatic_upgrade_enabled = true
auto_upgrade_minor_version = true
}
resource "azurerm_virtual_machine_extension" "msmonitor-agent" {
depends_on = [ azurerm_virtual_machine_extension.daa-agent ]
name = "MicrosoftMonitoringAgent" # Must be called this
virtual_machine_id = azurerm_windows_virtual_machine.windowsvm-c.id
publisher = "Microsoft.EnterpriseCloud.Monitoring"
type = "MicrosoftMonitoringAgent"
type_handler_version = "1.0"
# Not yet supported
# automatic_upgrade_enabled = true
# auto_upgrade_minor_version = true
settings = <<SETTINGS
{
"workspaceId": "${azurerm_log_analytics_workspace.law.id}",
"azureResourceId": "${azurerm_windows_virtual_machine.windowsvm-c.id}",
"stopOnMultipleConnections": "false"
}
SETTINGS
protected_settings = <<PROTECTED_SETTINGS
{
"workspaceKey": "${azurerm_log_analytics_workspace.law.primary_shared_key}"
}
PROTECTED_SETTINGS
}
Note the extended settings under "msmonitor-agent"
Here are few articles for this topic, maybe you can reference to:
Azure Monitor for application monitoring with Terraform
Azure Insights: Terraform; Log Analytics Workspaces; Custom scripts with Arc-enabled servers; Virtual WAN resources
My goal is to create an Ubuntu VM which will connect to aks without running az login.
The main idea behind that is that I want to let other people connect to that aks cluster only and not be able to read\write any other resources on Azure. Currently, I've tried to achieve that by creating a new role and assign this role to the VM, but, no luck so far.
My question is - is it possible to run az aks get-credentials ... without running az login?
Terraform template:
# Create AKS Cluster
resource "azurerm_kubernetes_cluster" "akscluster" {
count = var.cluster_count
name = "${var.cluster_name}-${count.index}"
location = var.location
resource_group_name = azurerm_resource_group.aksrg.name
dns_prefix = var.dns
default_node_pool {
name = var.node_pool_name
node_count = var.node_count
vm_size = var.vm_size
type = "VirtualMachineScaleSets"
}
service_principal {
client_id = var.kubernetes_client_id
client_secret = var.kubernetes_client_secret
}
tags = {
Environment = var.tags
}
}
# Create virtual machine
resource "azurerm_virtual_machine" "myterraformvm" {
count = var.cluster_count
name = "aks-${count.index}"
location = var.location
resource_group_name = azurerm_resource_group.aksrg.name
network_interface_ids = [azurerm_network_interface.myterraformnic.id]
vm_size = "Standard_DS1_v2"
storage_os_disk {
name = "myOsDisk"
caching = "ReadWrite"
create_option = "FromImage"
managed_disk_type = "Premium_LRS"
}
storage_image_reference {
publisher = "Canonical"
offer = "UbuntuServer"
sku = "18.04-LTS"
version = "latest"
}
os_profile {
computer_name = "hostname"
admin_username = "var.admin"
admin_password = "var.pass"
}
os_profile_linux_config {
disable_password_authentication = false
# ssh_keys {
# path = "/home/azureuser/.ssh/authorized_keys"
# key_data = "ssh-rsa AAAAB3Nz{snip}hwhqT9h"
# }
}
identity {
type = "SystemAssigned"
}
boot_diagnostics {
enabled = "true"
storage_uri = azurerm_storage_account.mystorageaccount.primary_blob_endpoint
}
tags = {
environment = var.tags
}
}
resource "azurerm_virtual_machine_extension" "example" {
count = var.cluster_count
name = "hostname"
virtual_machine_id = azurerm_virtual_machine.myterraformvm[count.index].id
publisher = "Microsoft.Azure.Extensions"
type = "CustomScript"
type_handler_version = "2.0"
settings = <<SETTINGS
{
"commandToExecute": "curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash"
}
SETTINGS
tags = {
environment = var.tags
}
}
You can create a remote file using the remote-exec provisioner, passing the azurerm_kubernetes_cluster.aks.kube_config_raw resource. I made an example for you here: https://github.com/ams0/terraform-templates/tree/master/aks-vm.
It creates a vnet with 2 subnets, an AKS in one and an ubuntu VM in the other one, and creates a local /home/ubuntu/.kube/config in the VM. You just need to download kubectl and you're good to go.
I am trying to encrypt the "storage_os_disk" on an Azure VM via Terraform.
I have set the managed disk type on the VM OS Disk, so it will be managed, since I know the disk must be managed to allow encryption.
I cannot seem to figure out how to encrypt the OS disk, in terraform
Here is my code i am trying:
resource "azurerm_network_interface" "nic" {
name = "${var.project_ident}-${var.env_ident}-${var.admin_vm_name}-${var.region_suffix}-encrpytest"
location = "${data.azurerm_resource_group.core-rg.location}"
resource_group_name = "${data.azurerm_resource_group.core-rg.name}"
depends_on = ["azurerm_virtual_machine.dns-vm"]
ip_configuration {
name = "${var.project_ident}-${var.env_ident}-${var.admin_vm_name}-${var.region_suffix}-encrpytest"
subnet_id ="${data.terraform_remote_state.network.sn1_id}"
private_ip_address_allocation = "static"
private_ip_address = "${cidrhost(data.terraform_remote_state.network.sn1_address_prefix, 6 )}"
}
}
resource "azurerm_virtual_machine" "admin-vm-encrpytest" {
name = "${var.project_ident}-${var.env_ident}-${var.admin_vm_name}-encrpytest"
location = "${data.azurerm_resource_group.core-rg.location}"
resource_group_name = "${data.azurerm_resource_group.core-rg.name}"
network_interface_ids = ["${azurerm_network_interface.nic.id}"]
vm_size = "Standard_B2s"
depends_on = ["azurerm_virtual_machine.dns-vm"]
# Requires LRS Storage Account
boot_diagnostics {
enabled = "True"
storage_uri = "${data.terraform_remote_state.sa.sa_2_prim_blob_ep}"
#storage_uri = "${data.azurerm_storage_account.storage-account-2.primary_blob_endpoint}"
}
storage_os_disk {
name = "${var.project_ident}-${var.env_ident}-${var.admin_vm_name}-${var.region_suffix}-encrpytest"
create_option = "FromImage"
managed_disk_type = "Standard_LRS"
encryption_settings {
enabled = "True"
key_encryption_key {
key_url = "${data.terraform_remote_state.kv.vault_key_1_id}"
source_vault_id = "${data.terraform_remote_state.kv.vault_id}"
}
disk_encryption_key {
secret_url = "${data.terraform_remote_state.kv.vault_key_2_id}"
source_vault_id = "${data.terraform_remote_state.kv.vault_id}"
}
}
}
os_profile {
computer_name = "encrpytest"
admin_username = "cactusadmin"
admin_password = "${var.admin_vm_password}"
}
os_profile_windows_config {
provision_vm_agent = true
enable_automatic_upgrades = true
}
# Uncomment this line to delete the OS disk automatically when deleting the VM
delete_os_disk_on_termination = true
# Uncomment this line to delete the data disks automatically when deleting the VM
delete_data_disks_on_termination = true
storage_image_reference {
publisher = "MicrosoftWindowsServer"
offer = "WindowsServer"
sku = "2016-Datacenter"
version = "latest"
}
}
Thank you
Firstly, the encryption_settings does not exist in the storage_os_disk block but azurerm_managed_disk. So you could create an individual azurerm_managed_disk resource then create VM from a managed disk with the platform image referring here.
Alternatively, you could try to use azurerm_virtual_machine_extension for disk-encryption, refer to this.
resource "azurerm_virtual_machine_extension" "disk-encryption" {
name = "DiskEncryption"
location = "${local.location}"
resource_group_name = "${azurerm_resource_group.environment-rg.name}"
virtual_machine_name = "${azurerm_virtual_machine.server.name}"
publisher = "Microsoft.Azure.Security"
type = "AzureDiskEncryption"
type_handler_version = "2.2"
settings = <<SETTINGS
{
"EncryptionOperation": "EnableEncryption",
"KeyVaultURL": "https://${local.vaultname}.vault.azure.net",
"KeyVaultResourceId": "/subscriptions/${local.subscriptionid}/resourceGroups/${local.vaultresourcegroup}/providers/Microsoft.KeyVault/vaults/${local.vaultname}",
"KeyEncryptionKeyURL": "https://${local.vaultname}.vault.azure.net/keys/${local.keyname}/${local.keyversion}",
"KekVaultResourceId": "/subscriptions/${local.subscriptionid}/resourceGroups/${local.vaultresourcegroup}/providers/Microsoft.KeyVault/vaults/${local.vaultname}",
"KeyEncryptionAlgorithm": "RSA-OAEP",
"VolumeType": "All"
}
SETTINGS
}
I used the vm extension example, and it worked perfectly. The OS disk on my newly deployed Windows VM, was instantly encrypted