Terraform default script path - terraform

I am creating multiple VMs in Azure using cloud-init, they are created in parallel and when any of them fails, I can see in the logs:
Error: error executing "/tmp/terraform_876543210.sh": Process exited with status 1
But I have no way to figure out which VM is failing, I need to ssh each of them and check
The script path seems to be defined for provisioning Terraform
Is there a way to override it also for cloud-init to something like: /tmp/terraform_vmName_876543210.sh ?
I am not using provisioner but cloud-init, any idea how I can force terraform to override the terraform sh file?
Below my script:
resource "azurerm_linux_virtual_machine" "example" {
name = "example-machine"
resource_group_name = azurerm_resource_group.example.name
location = azurerm_resource_group.example.location
size = "Standard_F2"
admin_username = "adminuser"
network_interface_ids = [
azurerm_network_interface.example.id,
]
admin_ssh_key {
username = "adminuser"
public_key = file("~/.ssh/id_rsa.pub")
}
os_disk {
caching = "ReadWrite"
storage_account_type = "Standard_LRS"
}
source_image_reference {
publisher = "Canonical"
offer = "UbuntuServer"
sku = "16.04-LTS"
version = "latest"
}
custom_data = base64encode(templatefile(
"my-cloud-init.tmpl", {
var1 = "value1"
var2 = "value2"
})
)
}
And my cloud-init script:
## template: jinja
#cloud-config
runcmd:
- sudo /tmp/bootstrap.sh
write_files:
- path: /tmp/bootstrap.sh
permissions: 00700
content: |
#!/bin/sh -e
echo hello

From the code you found for the Terraform, it shows:
DefaultUnixScriptPath is used as the path to copy the file to for
remote execution on Unix if not provided otherwise.
It's the configuration for the remote execution. And for the remote execution of the SSH, you can set the source and the destination for the copy file in the provisioner "file".
But it's used to set the path in the remote VM, not the local machine that you execute the Terraform. And you can overwrite the file name like this:
provisioner "file" {
source = "conf/myapp.conf"
destination = "/etc/terraform_$(var.vmName).conf"
...
}

Related

Read file and save output to local_file

I'm trying to read the content of a file on an azurerm_linux_virtual_machine and save it to a local_file so that an ansible playbook can reference it later. Currently the .tf looks like this
resource "azurerm_linux_virtual_machine" "vm" {
name = myvm
location = myzone
resource_group_name = azurerm_resource_group.azureansibledemo.name
network_interface_ids = [azurerm_network_interface.myterraformnic.id]
size = "Standard_DS1_v2"
os_disk {
name = "storage"
caching = "ReadWrite"
storage_account_type = "Premium_LRS"
}
source_image_reference {
publisher = "Canonical"
offer = "UbuntuServer"
sku = "16.04-LTS"
version = "latest"
}
computer_name = myvm
admin_username = "azureuser"
disable_password_authentication = true
custom_data = base64encode(file("telnet.sh"))
admin_ssh_key {
username = "azureuser"
public_key = tls_private_key.ansible_ssh_key.public_key_openssh
}
boot_diagnostics {
storage_account_uri = azurerm_storage_account.mystorageaccount.primary_blob_endpoint
}
}
output "myoutput" {
value = file("/tmp/output.yml")
}
resource "local_file" "testoutput" {
content = <<-DOC
${file("/tmp/output.yml")}
DOC
filename = "test.yml"
}
But when i run terraform plan i get the following error
Error: Invalid function argument
on main.tf line 181, in resource "local_file" "testoutput":
181: ${file("/tmp/output.yml")}
Invalid value for "path" parameter: no file exists at /tmp/output.yml; this
function works only with files that are distributed as part of the
configuration source code, so if this file will be created by a resource in
this configuration you must instead obtain this result from an attribute of
that resource.
The output myoutput is fine and returns no errors, this only occurs when i add in the resource local_file. Is there a way to get the output of a file to a local_file?
Copping remote files to local is not supported by TF.
The workaround is to use scp in local-exec as shown here. For example:
provisioner "local-exec" {
command = "scp -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ${var.openstack_keypair} ubuntu#${openstack_networking_floatingip_v2.wr_manager_fip.address}:~/client.token ."
}

Error handling in cloud init scripts within azurerm_linux_virtual_machine

I run a custom shell script when I deploy my virtual machine with terraform, which can throw errors.
My question is, how do you handle these errors, because regardless of the return code of the script, terraform always reports the deployment was successful, which leads to confusion when the VM does not what it’s supposed to do.
Here a snippet of the terraform file for context:
data "template_file" "setup_script" {
count = var.agent_count
template = file("scripts/setup.sh")
vars = {
POOL_NAME = var.pool_name
AGENT = "agent-${count.index}"
ORGANIZATION_URL = var.organization_url
TOKEN = var.token
TERRAFORM_VERSION = var.terraform_version
VSTS_AGENT_VERSION = var.vsts_agent_version
}
}
resource "azurerm_linux_virtual_machine" "vmachine" {
count = length(module.network.network_interfaces)
name = "agent-${count.index}"
resource_group_name = azurerm_resource_group.deployment-agents.name
location = azurerm_resource_group.deployment-agents.location
size = "Standard_B1ms"
admin_username = "adminuser"
network_interface_ids = [
module.network.network_interfaces[count.index].id,
]
admin_ssh_key {
username = "adminuser"
public_key = var.ssh_public_key
}
os_disk {
caching = "ReadWrite"
storage_account_type = "Standard_LRS"
}
source_image_reference {
publisher = "Canonical"
offer = "UbuntuServer"
sku = "18.04-LTS"
version = "latest"
}
boot_diagnostics {
storage_account_uri = azurerm_storage_account.boot.primary_blob_endpoint
}
custom_data = base64encode(data.template_file.setup_script.*.rendered[count.index])
}
And the setup.sh shell script:
# --- snip ----
apt-get install azure-cli
if [ $? -gt 0 ]; then
echo "Cannot install azure cli!"
exit 1
fi
# Test
exit 1
Thanks for the help.
Terraform only return the error about itself, not the script execute inside the VM. You can find the error message inside the VM.
And to install the Azure CLI via the cloud-init with a shell script, you need to add #!/bin/bash at the beginning of the shell script, see the note:
And install the Azure CLI, I think there are more things you need to do than what you have tried, take a look at the steps that install the Azure CLI in Ubuntu. Or use the existing shell script here.
The important piece here is "a cloud-init image deployment will NOT fail if the script fails" from the Azure cloud-init docs.
This mean that Azure think the deployment was successful, which is what Terraform will also show.
In order to fail the Terraform deployment, you would have to use cloud-init to install the cli using cloud-init itself, which should make the deployment fail:
data "template_cloudinit_config" "config" {
gzip = true
base64_encode = true
part {
content_type = "text/cloud-config"
content = "packages: ['azure-cli']"
}
}
More examples on what could be done using cloud-init in the docs.

Terraform Azure run bash script on VM

I'm trying to run a bash script on an Azure VM after deploying it with Terraform. I've tried different approaches but none of them have worked. With "custom_data", I assumed that the file will be uploaded and executed, however I'm not even seeing the file inside the VM.
I've also looked at "azurerm_virtual_machine_extension", but this does not give me the option to upload the file, only to execute commands or download from remote location (can't use fileUris due to requirements):
resource "azurerm_virtual_machine_extension" "test" {
name = "hostname"
location = "${azurerm_resource_group.test.location}"
resource_group_name = "${azurerm_resource_group.test.name}"
virtual_machine_name = "${azurerm_virtual_machine.test.name}"
publisher = "Microsoft.Azure.Extensions"
type = "CustomScript"
type_handler_version = "2.0"
settings = <<SETTINGS
{
"commandToExecute": "sh my_script.sh"
}
SETTINGS
tags = {
environment = "Production"
}
}
resource "azurerm_virtual_machine" "middleware_vm" {
name = "${var.middleware_vm}"
location = "${var.location}"
resource_group_name = "${azurerm_resource_group.middleware.name}"
network_interface_ids = ["${azurerm_network_interface.middleware.id}"]
vm_size = "Standard_DS4_v2"
storage_os_disk {
name = "centos_os_disk"
caching = "ReadWrite"
create_option = "FromImage"
managed_disk_type = "Premium_LRS"
}
storage_data_disk {
name = "managed_backup_disk"
create_option = "Empty"
caching = "ReadWrite"
disk_size_gb = "256"
managed_disk_type = "Premium_LRS"
lun = 0
}
storage_image_reference {
publisher = "OpenLogic"
offer = "CentOS"
sku = "7.5"
version = "latest"
}
os_profile {
computer_name = "${var.middleware_vm}"
admin_username = "middlewareadmin"
custom_data = "${file("scripts/middleware_disk.sh")}"
}
In azurerm_virtual_machine_extension, you can use:
protected_settings = <<PROTECTED_SETTINGS
{
"script": "${base64encode(file(var.scfile))}"
}
PROTECTED_SETTINGS
Please refer to my answer
First, the VM extension will just execute the script and do not copy the file to the VM. If you want to copy the script into the VM and then execute it. I will suggest you the Terraform provisioner file and remote-exec.
Here is the example that copies the file into the existing VM and executes the script:
resource "null_resource" "example" {
connection {
type = "ssh"
user = "azureuser"
password = "azureuser#2018"
host = "13.92.255.50"
port = 22
}
provisioner "file" {
source = "script.sh"
destination = "/tmp/script.sh"
}
provisioner "remote-exec" {
inline = [
"/bin/bash /tmp/script.sh"
]
}
}
Note: the script should be created in the current directory.

How can I use Terraform's file provisioner to copy from my local machine onto a VM?

I'm new to Terraform and have so far managed to get a basic VM (plus Resource Manager trimmings) up and running on Azure. The next task I have in mind is to have Terraform copy a file from my local machine into the newly created instance. Ideally I'm after a solution where the file will be copied each time the apply command is run.
I feel like I'm pretty close but so far I just get endless "Still creating..." statements once I apply (the file is 0kb so after a couple of mins it feels safe to give up).
So far, this is what I've got (based on this code): https://stackoverflow.com/a/37866044/4941009
Network
resource "azurerm_public_ip" "pub-ip" {
name = "PublicIp"
location = "${var.location}"
resource_group_name = "${azurerm_resource_group.rg.name}"
public_ip_address_allocation = "Dynamic"
domain_name_label = "${var.hostname}"
}
VM
resource "azurerm_virtual_machine" "vm" {
name = "${var.hostname}"
location = "${var.location}"
resource_group_name = "${azurerm_resource_group.rg.name}"
vm_size = "${var.vm_size}"
network_interface_ids = ["${azurerm_network_interface.nic.id}"]
storage_image_reference {
publisher = "${var.image_publisher}"
offer = "${var.image_offer}"
sku = "${var.image_sku}"
version = "${var.image_version}"
}
storage_os_disk {
name = "${var.hostname}osdisk1"
vhd_uri = "${azurerm_storage_account.stor.primary_blob_endpoint}${azurerm_storage_container.cont.name}/${var.hostname}osdisk.vhd"
os_type = "${var.os_type}"
caching = "ReadWrite"
create_option = "FromImage"
}
os_profile {
computer_name = "${var.hostname}"
admin_username = "${var.admin_username}"
admin_password = "${var.admin_password}"
}
os_profile_windows_config {
provision_vm_agent = true
}
boot_diagnostics {
enabled = true
storage_uri = "${azurerm_storage_account.stor.primary_blob_endpoint}"
}
tags {
environment = "${var.environment}"
}
}
File Provisioner
resource "null_resource" "copy-test-file" {
connection {
type = "ssh"
host = "${azurerm_virtual_machine.vm.ip_address}"
user = "${var.admin_username}"
password = "${var.admin_password}"
}
provisioner "file" {
source = "test.txt"
destination = "/tmp/test.txt"
}
}
As an aside, if I pass incorrect login details to the provisioner (ie rerun this after the VM has already been created and supply a different password to the provisioner) the behaviour is the same. Can anyone suggest where I'm going wrong?
I eventually got this working. Honestly, I kind of forgot about this question so I can't remember what my exact issue was, the example below though seems to work on my instance:
resource "null_resource" remoteExecProvisionerWFolder {
provisioner "file" {
source = "test.txt"
destination = "/tmp/test.txt"
}
connection {
host = "${azurerm_virtual_machine.vm.ip_address}"
type = "ssh"
user = "${var.admin_username}"
password = "${var.admin_password}"
agent = "false"
}
}
So it looks like the only difference here is the addition of agent = "false". This would make some sense as there's only one SSH authentication agent for windows and it's probable I hadn't explicitly specified to use that agent before. However it could well be that I ultimately changed something elsewhere in the configuration. Sorry future people for not being much help on this one.
FYI, windows instances you can connect via type winrm
resource "null_resource" "provision_web" {
connection {
host = "${azurerm_virtual_machine.vm.ip_address}"
type = "winrm"
user = "alex"
password = "alexiscool1!"
}
provisioner "file" {
source = "path/to/folder"
destination = "C:/path/to/destination"
}
}

SSH connection to Azure VM with Terraform

I have successfully created a VM as part of a Resource Group on Azure using Terraform. Next step is to ssh in the new machine and run a few commands. For that, I have created a provisioner as part of the VM resource and set up an SSH connection:
resource "azurerm_virtual_machine" "helloterraformvm" {
name = "terraformvm"
location = "West US"
resource_group_name = "${azurerm_resource_group.helloterraform.name}"
network_interface_ids = ["${azurerm_network_interface.helloterraformnic.id}"]
vm_size = "Standard_A0"
storage_image_reference {
publisher = "Canonical"
offer = "UbuntuServer"
sku = "14.04.2-LTS"
version = "latest"
}
os_profile {
computer_name = "hostname"
user = "some_user"
password = "some_password"
}
os_profile_linux_config {
disable_password_authentication = false
}
provisioner "remote-exec" {
inline = [
"sudo apt-get install docker.io -y"
]
connection {
type = "ssh"
user = "some_user"
password = "some_password"
}
}
}
If I run "terraform apply", it seems to get into an infinite loop trying to ssh unsuccessfully, repeating this log over and over:
azurerm_virtual_machine.helloterraformvm (remote-exec): Connecting to remote host via SSH...
azurerm_virtual_machine.helloterraformvm (remote-exec): Host:
azurerm_virtual_machine.helloterraformvm (remote-exec): User: testadmin
azurerm_virtual_machine.helloterraformvm (remote-exec): Password: true
azurerm_virtual_machine.helloterraformvm (remote-exec): Private key: false
azurerm_virtual_machine.helloterraformvm (remote-exec): SSH Agent: true
I'm sure I'm doing something wrong, but I don't know what it is :(
EDIT:
I have tried setting up this machine without the provisioner, and I can SSH to it no problems with the given username/passwd. However I need to look up the host name in the Azure portal because I don't know how to retrieve it from Terraform. It's suspicious that the "Host:" line in the log is empty, so I wonder if it has anything to do with that?
UPDATE:
I've tried with different things like indicating the host name in the connection with
host = "${azurerm_public_ip.helloterraformip.id}"
and
host = "${azurerm_public_ip.helloterraformips.ip_address}"
as indicated in the docs, but with no success.
I've also tried using ssh-keys instead of password, but same result - infinite loop of connection tries, with no clear error message as of why it's not connecting.
I have managed to make this work. I changed several things:
Gave name of host to connection.
Configured SSH keys properly - they need to be unencrypted.
Took the connection element out of the provisioner element.
Here's the full working Terraform file, replacing the data like SSH keys, etc.:
# Configure Azure provider
provider "azurerm" {
subscription_id = "${var.azure_subscription_id}"
client_id = "${var.azure_client_id}"
client_secret = "${var.azure_client_secret}"
tenant_id = "${var.azure_tenant_id}"
}
# create a resource group if it doesn't exist
resource "azurerm_resource_group" "rg" {
name = "sometestrg"
location = "ukwest"
}
# create virtual network
resource "azurerm_virtual_network" "vnet" {
name = "tfvnet"
address_space = ["10.0.0.0/16"]
location = "ukwest"
resource_group_name = "${azurerm_resource_group.rg.name}"
}
# create subnet
resource "azurerm_subnet" "subnet" {
name = "tfsub"
resource_group_name = "${azurerm_resource_group.rg.name}"
virtual_network_name = "${azurerm_virtual_network.vnet.name}"
address_prefix = "10.0.2.0/24"
#network_security_group_id = "${azurerm_network_security_group.nsg.id}"
}
# create public IPs
resource "azurerm_public_ip" "ip" {
name = "tfip"
location = "ukwest"
resource_group_name = "${azurerm_resource_group.rg.name}"
public_ip_address_allocation = "dynamic"
domain_name_label = "sometestdn"
tags {
environment = "staging"
}
}
# create network interface
resource "azurerm_network_interface" "ni" {
name = "tfni"
location = "ukwest"
resource_group_name = "${azurerm_resource_group.rg.name}"
ip_configuration {
name = "ipconfiguration"
subnet_id = "${azurerm_subnet.subnet.id}"
private_ip_address_allocation = "static"
private_ip_address = "10.0.2.5"
public_ip_address_id = "${azurerm_public_ip.ip.id}"
}
}
# create storage account
resource "azurerm_storage_account" "storage" {
name = "someteststorage"
resource_group_name = "${azurerm_resource_group.rg.name}"
location = "ukwest"
account_type = "Standard_LRS"
tags {
environment = "staging"
}
}
# create storage container
resource "azurerm_storage_container" "storagecont" {
name = "vhd"
resource_group_name = "${azurerm_resource_group.rg.name}"
storage_account_name = "${azurerm_storage_account.storage.name}"
container_access_type = "private"
depends_on = ["azurerm_storage_account.storage"]
}
# create virtual machine
resource "azurerm_virtual_machine" "vm" {
name = "sometestvm"
location = "ukwest"
resource_group_name = "${azurerm_resource_group.rg.name}"
network_interface_ids = ["${azurerm_network_interface.ni.id}"]
vm_size = "Standard_A0"
storage_image_reference {
publisher = "Canonical"
offer = "UbuntuServer"
sku = "16.04-LTS"
version = "latest"
}
storage_os_disk {
name = "myosdisk"
vhd_uri = "${azurerm_storage_account.storage.primary_blob_endpoint}${azurerm_storage_container.storagecont.name}/myosdisk.vhd"
caching = "ReadWrite"
create_option = "FromImage"
}
os_profile {
computer_name = "testhost"
admin_username = "testuser"
admin_password = "Password123"
}
os_profile_linux_config {
disable_password_authentication = false
ssh_keys = [{
path = "/home/testuser/.ssh/authorized_keys"
key_data = "ssh-rsa xxx email#something.com"
}]
}
connection {
host = "sometestdn.ukwest.cloudapp.azure.com"
user = "testuser"
type = "ssh"
private_key = "${file("~/.ssh/id_rsa_unencrypted")}"
timeout = "1m"
agent = true
}
provisioner "remote-exec" {
inline = [
"sudo apt-get update",
"sudo apt-get install docker.io -y",
"git clone https://github.com/somepublicrepo.git",
"cd Docker-sample",
"sudo docker build -t mywebapp .",
"sudo docker run -d -p 5000:5000 mywebapp"
]
}
tags {
environment = "staging"
}
}
According to your description, Azure Custom Script Extension is an option for you.
The Custom Script Extension downloads and executes scripts on Azure
virtual machines. This extension is useful for post deployment
configuration, software installation, or any other configuration /
management task.
Remove provisioner "remote-exec" instead of below:
resource "azurerm_virtual_machine_extension" "helloterraformvm" {
name = "hostname"
location = "West US"
resource_group_name = "${azurerm_resource_group.helloterraformvm.name}"
virtual_machine_name = "${azurerm_virtual_machine.helloterraformvm.name}"
publisher = "Microsoft.OSTCExtensions"
type = "CustomScriptForLinux"
type_handler_version = "1.2"
settings = <<SETTINGS
{
"commandToExecute": "apt-get install docker.io -y"
}
SETTINGS
}
Note: Command is executed by root user, don't use sudo.
More information please refer to this link: azurerm_virtual_machine_extension.
For a list of possible extensions, you can use the Azure CLI command az vm extension image list -o table
Update: The above example only supports single command. If you need to multiple commands. Like install docker on your VM, you need
apt-get update
apt-get install docker.io -y
Save it as a file named script.sh and save it to Azure Storage account or GitHub(The file should be public). Modify terraform file like below:
settings = <<SETTINGS
{
"fileUris": ["https://gist.githubusercontent.com/Walter-Shui/dedb53f71da126a179544c91d267cdce/raw/bb3e4d90e3291530570eca6f4ff7981fdcab695c/script.sh"],
"commandToExecute": "sh script.sh"
}
SETTINGS

Resources