Terraform: SSH authentication failed (user#:22): ssh: handshake failed - azure

Question

I wrote some Terraform code to create a new VM and want to execute a command on it via remote-exec but it throws an SSH connection error:
Error: timeout - last error: SSH authentication failed (admin#:22): ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain.
My Terraform code:
# Create a resource group if it doesn’t exist
resource "azurerm_resource_group" "rg" {
name = "${var.deployment}-mp-rg"
location = "${var.azure_environment}"
tags = {
environment = "${var.deployment}"
}
}
# Create virtual network
resource "azurerm_virtual_network" "vnet" {
name = "${var.deployment}-mp-vnet"
address_space = ["10.0.0.0/16"]
location = "${var.azure_environment}"
resource_group_name = "${azurerm_resource_group.rg.name}"
tags = {
environment = "${var.deployment}"
}
}
# Create subnet
resource "azurerm_subnet" "subnet" {
name = "${var.deployment}-mp-subnet"
resource_group_name = "${azurerm_resource_group.rg.name}"
virtual_network_name = "${azurerm_virtual_network.vnet.name}"
address_prefix = "10.0.1.0/24"
}
# Create public IPs
resource "azurerm_public_ip" "publicip" {
name = "${var.deployment}-mp-publicip"
location = "${var.azure_environment}"
resource_group_name = "${azurerm_resource_group.rg.name}"
allocation_method = "Dynamic"
tags = {
environment = "${var.deployment}"
}
}
# Create Network Security Group and rule
resource "azurerm_network_security_group" "nsg" {
name = "${var.deployment}-mp-nsg"
location = "${var.azure_environment}"
resource_group_name = "${azurerm_resource_group.rg.name}"
security_rule {
name = "SSH"
priority = 1001
direction = "Inbound"
access = "Allow"
protocol = "Tcp"
source_port_range = "*"
destination_port_range = "22"
source_address_prefix = "*"
destination_address_prefix = "*"
}
tags = {
environment = "${var.deployment}"
}
}
# Create network interface
resource "azurerm_network_interface" "nic" {
name = "${var.deployment}-mp-nic"
location = "${var.azure_environment}"
resource_group_name = "${azurerm_resource_group.rg.name}"
network_security_group_id = "${azurerm_network_security_group.nsg.id}"
ip_configuration {
name = "${var.deployment}-mp-nicconfiguration"
subnet_id = "${azurerm_subnet.subnet.id}"
private_ip_address_allocation = "Dynamic"
public_ip_address_id = "${azurerm_public_ip.publicip.id}"
}
tags = {
environment = "${var.deployment}"
}
}
# Generate random text for a unique storage account name
resource "random_id" "randomId" {
keepers = {
# Generate a new ID only when a new resource group is defined
resource_group = "${azurerm_resource_group.rg.name}"
}
byte_length = 8
}
# Create storage account for boot diagnostics
resource "azurerm_storage_account" "storageaccount" {
name = "diag${random_id.randomId.hex}"
resource_group_name = "${azurerm_resource_group.rg.name}"
location = "${var.azure_environment}"
account_tier = "Standard"
account_replication_type = "LRS"
tags = {
environment = "${var.deployment}"
}
}
# Create virtual machine
resource "azurerm_virtual_machine" "vm" {
name = "${var.deployment}-mp-vm"
location = "${var.azure_environment}"
resource_group_name = "${azurerm_resource_group.rg.name}"
network_interface_ids = ["${azurerm_network_interface.nic.id}"]
vm_size = "Standard_DS1_v2"
storage_os_disk {
name = "${var.deployment}-mp-disk"
caching = "ReadWrite"
create_option = "FromImage"
managed_disk_type = "Premium_LRS"
}
storage_image_reference {
publisher = "Canonical"
offer = "UbuntuServer"
sku = "16.04-LTS"
version = "latest"
}
os_profile {
computer_name = "${var.deployment}-mp-ansible"
admin_username = "${var.ansible_user}"
}
os_profile_linux_config {
disable_password_authentication = true
ssh_keys {
path = "/home/${var.ansible_user}/.ssh/authorized_keys"
key_data = "${var.public_key}"
}
}
boot_diagnostics {
enabled = "true"
storage_uri = "${azurerm_storage_account.storageaccount.primary_blob_endpoint}"
}
tags = {
environment = "${var.deployment}"
}
}
resource "null_resource" "ssh_connection" {
connection {
host = "${azurerm_public_ip.publicip.ip_address}"
type = "ssh"
private_key = "${file(var.private_key)}"
port = 22
user = "${var.ansible_user}"
agent = false
timeout = "1m"
}
provisioner "remote-exec" {
inline = ["sudo apt-get -qq install python"]
}
}
I have tried to SSH into the new VM manually with admin#xx.xx.xx.xx:22 and it works. Looking at the error message I then output the parameter ${azurerm_public_ip.publicip.ip_address} but it is null so I think that this is the reason why the SSH authentication failed but I don't know the reason. If I want to SSH the server via Terraform script, how can I modify the code?

Answer 1

Your issue is that Terraform has built a dependency graph that tells it that the only dependency for the null_resource.ssh_connection is the azurerm_public_ip.publicip resource and so it's starting to try to connect before the instance has been created.
This in itself isn't a massive issues as the provisioner would normally attempt to retry in case SSH isn't yet available but the connection details are being determined as soon as the null resource starts. And with the azurerm_public_ip set to an allocation_method of Dynamic it won't get its IP address until after it has been attached to a resource:
Note Dynamic Public IP Addresses aren't allocated until they're assigned to a resource (such as a Virtual Machine or a Load Balancer) by design within Azure - more information is available below.
There's a few different ways you can solve this. You could make the null_resource depend on the azurerm_virtual_machine.vm resource via interpolation or via depends_on:
resource "null_resource" "ssh_connection" {
connection {
host = "${azurerm_public_ip.publicip.ip_address}"
type = "ssh"
private_key = "${file(var.private_key)}"
port = 22
user = "${var.ansible_user}"
agent = false
timeout = "1m"
}
provisioner "remote-exec" {
inline = [
"echo ${azurerm_virtual_machine.vm.id}",
"sudo apt-get -qq install python",
]
}
}
or
resource "null_resource" "ssh_connection" {
depends_on = ["azurerm_virtual_machine.vm"]
connection {
host = "${azurerm_public_ip.publicip.ip_address}"
type = "ssh"
private_key = "${file(var.private_key)}"
port = 22
user = "${var.ansible_user}"
agent = false
timeout = "1m"
}
provisioner "remote-exec" {
inline = ["sudo apt-get -qq install python"]
}
}
A better approach here would to be to run the provisioner as part of the azurerm_virtual_machine.vm resource instead of a null_resource. The normal reasons to use a null_resource to launch a provisioner are when you need to wait until after something else has happened to a resource such as attaching a disk or if there's not an appropriate resource to attach it to but this doesn't really apply here. So instead of your existing null_resource you'd move the provisioner into the azurerm_virtual_machine.vm resource:
resource "azurerm_virtual_machine" "vm" {
# ...
provisioner "remote-exec" {
connection {
host = "${azurerm_public_ip.publicip.ip_address}"
type = "ssh"
private_key = "${file(var.private_key)}"
port = 22
user = "${var.ansible_user}"
agent = false
timeout = "1m"
}
inline = ["sudo apt-get -qq install python"]
}
}
For many resources this also allows you to refer to the outputs of the resource you are provisioning by using the self keyword. Unfortunately the azurerm_virtual_machine resource doesn't seem to easily expose the IP address of the VM due to this being set by the network_interface_ids.