I tried to use the remote-exec to execute several commands on target VM, but failed with 'bash: Permission denied', here is the code:
connection {
host = "${azurerm_network_interface.nic.private_ip_address}"
type = "ssh"
user = "${var.mp_username}"
private_key = "${file(var.mp_vm_private_key)}"
}
provisioner "remote-exec" {
inline = [
"sudo wget https://raw.githubusercontent.com/Microsoft/OMS-Agent-for-Linux/master/installer/scripts/onboard_agent.sh",
"sudo chown ${var.mp_username}: onboard_agent.sh",
"sudo chmod +x onboard_agent.sh",
"./onboard_agent.sh -w ${azurerm_log_analytics_workspace.workspace.workspace_id} -s ${azurerm_log_analytics_workspace.workspace.primary_shared_key} -d opinsights.azure.us"
]
}
After checked the issue here: https://github.com/hashicorp/terraform/issues/5397, I need to wrap all the commands into a file. Then I used a template file to put all the commands in it:
OMSAgent.sh
#!/bin/bash
sudo wget https://raw.githubusercontent.com/Microsoft/OMS-Agent-for-Linux/master/installer/scripts/onboard_agent.sh
sudo chown ${username}: onboard_agent.sh
sudo chmod +x onboard_agent.sh
./onboard_agent.sh -w ${workspaceId} -s ${workspaceKey} -d opinsights.azure.us
The code changes to:
data "template_file" "extension_data" {
template = "${file("templates/OMSAgent.sh")}"
vars = {
workspaceId = "${azurerm_log_analytics_workspace.workspace.workspace_id}"
workspaceKey = "${azurerm_log_analytics_workspace.workspace.primary_shared_key}"
username = "${var.mp_username}"
}
}
resource "null_resource" "remote-provisioner" {
connection {
host = "${azurerm_network_interface.nic.private_ip_address}"
type = "ssh"
user = "${var.mp_username}"
private_key = "${file(var.mp_vm_private_key)}"
script_path = "/home/${var.mp_username}/OMSAgent.sh"
}
provisioner "file" {
content = "${data.template_file.extension_data.rendered}"
destination = "/home/${var.mp_username}/OMSAgent.sh"
}
provisioner "remote-exec" {
inline = [
"chmod +x /home/${var.mp_username}/OMSAgent.sh",
"/home/${var.mp_username}/OMSAgent.sh"
]
}
}
But seems something wrong in the null_resource, the null resource installation stoped and throws this:
null_resource.remote-provisioner (remote-exec): /home/user/OMSAgent.sh: 2: /home/user/OMSAgent.sh: Cannot fork
.
And the content for the shell script is this:
cat OMSAgent.sh
#!/bin/sh
chmod +x /home/user/OMSAgent.sh
/home/user/OMSAgent.sh
Seems I did the script in the wrong way.
#joe huang Please make sure you use the username and password provided when you created the os_profile for your VM:
os_profile {
computer_name = "hostname"
admin_username = "testadmin"
admin_password = "Password1234!"
}
https://www.terraform.io/docs/providers/azurerm/r/virtual_machine.html#example-usage-from-an-azure-platform-image-
Here is a document for installing the OMS agent:
https://support.microsoft.com/en-in/help/4131455/how-to-reinstall-operations-management-suite-oms-agent-for-linux
Hope this helps!
if your /tmp is mounted with noexec, the default location that TF uses to push its tmp script needs to change, perhaps to your users home dir. In the connection block add:
script_path = "~/terraform_provisioner_%RAND%.sh"
Related
I am trying to create a script with terraform that sends a file contained in a folder and overwrites the old file on the server if it has changed since.
I succeeded in sending the file on the server, however when I do the command: "terraform plan" after having modified my file it tells me that my configuration has not changed, and I don't want to have to modify an environment variable,
I would like it to be done automatically, has anyone ever had to deal with this situation?
My try #1:
resource "null_resource" "example" {
provisioner "file" {
source = "${path.cwd}/conf/file"
destination = "/home/user/conf/file"
connection {
host = "IP"
user = "user"
private_key = file("C:\\Users\\user\\.ssh\\sshTerraformDeployment")
}
}
}
Try #2:
resource "null_resource" "example" {
provisioner "remote-exec" {
inline = [
"set -e",
"cd /home/user/conf",
"rsync --ignore-existing --checksum -avz -e 'ssh -i /root/.ssh/sshTerraformDeployment' ${path.cwd}/conf root#host:/home/user/conf"
]
connection {
host = "IP"
user = "user"
private_key = file("C:\\Users\\user\\.ssh\\sshTerraformDeployment")
}
}
}
I want to use count to install a package on 2 of my VMs using a single remote-exec provisioner. As of now, I am doing that individually in 2 provisioners blocks as below.
----present code to use remote provisioner for 2 vms-----
resource "null_resource" "install_nginx_host1" {
provisioner "remote-exec" {
inline = [
"sudo apt install nginx -y"
]
}
connection {
type = "ssh"
user = "ubuntu"
private_key = file("~/.ssh/id_rsa")
host = module.virtual-machine[0].linux_vm_public_ips.instance-0
}
}
resource "null_resource" "install_nginx_host2" {
provisioner "remote-exec" {
inline = [
"sudo apt install nginx -y"
]
}
connection {
type = "ssh"
user = "ubuntu"
private_key = file("~/.ssh/id_rsa")
host = module.virtual-machine[1].linux_vm_public_ips.instance-1
}
}
Can someone please help me in getting the value which I should should to set host using count.index? I tried multiple things e.g.
host = "module.virtual-machine[${count.index}].linux_vm_public_ips.instance-${count.index}"
But it returns the host strings as:
module.virtual-machine[0].linux_vm_public_ips.instance-0
module.virtual-machine[1].linux_vm_public_ips.instance-1
while I want the value of above strings.
This should be pretty straightforward to achieve:
resource "null_resource" "install_nginx_host1" {
count = 2
provisioner "remote-exec" {
inline = [
"sudo apt install nginx -y"
]
}
connection {
type = "ssh"
user = "ubuntu"
private_key = file("~/.ssh/id_rsa")
host = module.virtual-machine[count.index].linux_vm_public_ips["instance-${count.index}"]
}
}
Please make sure you understand how to use the count meta-argument [1].
[1] https://www.terraform.io/language/meta-arguments/count
While trying to provision with file multiple times, second occurance is not being considered. Not sure if I'm doing it correctly.
Please throw some light !
The below block works perfectly -
source = "/home/ubuntu/Desktop/aws_migration_using_terraform/tcs-btag-account_us-east-2/aws_infra_automation"
destination = "/home/ubuntu"
}
However, this one didn't work and there is no error thrown by terraform itself !
source = "/home/ubuntu/Desktop/aws_migration_using_terraform/tcs-btag-account_us-east-2/livedevops"
destination = "/home/ubuntu"
}
The entire code is given below --
resource "tls_private_key" "bastion-key" {
algorithm = "RSA"
rsa_bits = 4096
}
resource "aws_key_pair" "generated_key" {
key_name = var.bastion_key
public_key = tls_private_key.bastion-key.public_key_openssh
}
resource "aws_instance" "bastion_host_us-east-2a" {
ami = var.bastion_ami_id
instance_type = var.bastion_ec2_instance_type
disable_api_termination = false
subnet_id = aws_subnet.devops_mig_pub_sub_01.id
vpc_security_group_ids = [aws_security_group.sg-btag-allow.id, aws_security_group.sg-ssh-allow.id]
associate_public_ip_address = true
availability_zone = aws_subnet.devops_mig_pub_sub_01.availability_zone
key_name = aws_key_pair.generated_key.id
connection {
type = "ssh"
host = self.public_ip
user = "ubuntu"
port = 22
private_key = tls_private_key.bastion-key.private_key_pem
timeout = "60s"
}
#Copying files from local to remote
provisioner "file" {
source = "/home/ubuntu/Desktop/aws_migration_using_terraform/tcs-btag-account_us-east-2/aws_infra_automation"
destination = "/home/ubuntu"
}
provisioner "file" {
source = "/home/ubuntu/Desktop/aws_migration_using_terraform/tcs-btag-account_us-east-2/livedevops"
destination = "/home/ubuntu"
}
user_data = <<-EOF
#!/bin/bash
sudo apt update -y
sudo apt install -y software-properties-common
sudo add-apt-repository --yes --update ppa:ansible/ansible
sudo apt update -y
sudo apt install -y ansible
/usr/bin/ansible --version > ansible-v.txt
echo "Installing the cloudwatch agent for Ubuntu Linux."
curl -O https://s3.amazonaws.com/amazoncloudwatch-agent/ubuntu/amd64/latest/amazon-cloudwatch-agent.deb
dpkg -i -E ./amazon-cloudwatch-agent.deb
EOF
tags = {
"Name" = "bastion_host"
}
}
output "private_key" {
value = tls_private_key.bastion-key.private_key_pem
sensitive = true
}
output "bastion_public_ip" {
value = aws_instance.bastion_host_us-east-2a.public_ip
}
output "bastion_private_ip" {
value = aws_instance.bastion_host_us-east-2a.private_ip
}
resource "aws_ebs_volume" "bastion_storage" {
availability_zone = var.bastion-ebs-availability-zone
size = 50
type = "gp2"
tags = {
"Name" = "bastion_ebs_volume"
}
}
resource "local_file" "bastion_private_key" {
content = tls_private_key.bastion-key.private_key_pem
filename = "bastion-key.pem"
file_permission = "0400"
}
I see ubuntu being the user used to SSH to target machine. It's a bad idea to copy files directly to HOME directory of the user & in this case the file provisioner is just replacing everything available on /home/ubuntu directory.
The above directory also contains your SSH public keys used for authentication in ~/.ssh/authorized_keys. That's the reason it's failing at the second file provisioner.
You create a tmp directory under /home/ubuntu or use /tmp or /var/tmp directories if they allow ubuntu user to write something to write.
I am currently struggling to get the user_data script to run when starting the EC2 instance using Terraform. I pre-configured my AMI using Packer, and referenced the custom AMI in my Terraform file. Since I need to now the RDS instance URL when starting the EC2 instance, I tried to read them inside the user_data script and set them as environment variables. My app tries to read these environment variables and can connect to the db. Everything works as expected locally, and on CI when running tests. Manually setting the variables and starting the app also works as expected. The only problem is the execution of the user_data script because it already ran when creating the AMI using Packer.
Notice how I read the current DB state inside Terraform, which is why I cannot use traditional approaches that would result in the user_data script getting executed again. I also tried deleting the cloud data as described in this question and this one without success.
This is my current Packer configuration:
build {
name = "spring-ubuntu"
sources = [
"source.amazon-ebs.ubuntu"
]
provisioner "file" {
source = "build/libs/App.jar"
destination = "~/App.jar"
}
provisioner "shell" {
inline = [
"sleep 30",
"sudo apt update",
"sudo apt -y install openjdk-17-jdk",
"sudo rm -Rf /var/lib/cloud/data/scripts",
"sudo rm -Rf /var/lib/cloud/scripts/per-instance",
"sudo rm -Rf /var/lib/cloud/data/user-data*",
"sudo rm -Rf /var/lib/cloud/instances/*",
"sudo rm -Rf /var/lib/cloud/instance",
]
}
}
This is my current Terraform configuration:
resource "aws_instance" "instance" {
ami = "ami-123abc"
instance_type = "t2.micro"
subnet_id = tolist(data.aws_subnet_ids.all.ids)[0]
vpc_security_group_ids = [aws_security_group.ec2.id]
user_data = <<EOF
#!/bin/bash
export DB_HOST=${data.terraform_remote_state.state.outputs.db_address}
export DB_PORT=${data.terraform_remote_state.state.outputs.db_port}
java -jar ~/App.jar
EOF
lifecycle {
create_before_destroy = true
}
}
I tried to run the commands using the remote-exec provisioner as proposed by Marko E. At first it failed with the following error, but after a second try, it worked.
This object does not have an attribute named "db_address".
This object does not have an attribute named "db_port".
This is the working configuration
resource "aws_instance" "instance" {
ami = "ami-0ed33809ce5c950b9"
instance_type = "t2.micro"
key_name = "myKeys"
subnet_id = tolist(data.aws_subnet_ids.all.ids)[0]
vpc_security_group_ids = [aws_security_group.ec2.id]
lifecycle {
create_before_destroy = true
}
connection {
type = "ssh"
user = "ubuntu"
private_key = file("~/Downloads/myKeys.pem")
host = self.public_ip
}
provisioner "remote-exec" {
inline = [
"export DB_HOST=${data.terraform_remote_state.state.outputs.db_address}",
"export DB_PORT=${data.terraform_remote_state.state.outputs.db_port}",
"java -jar ~/App.jar &",
]
}
}
I am trying to set up a group of EC2s for an app using Terraform in AWS. After each server is created I want to mount the eNVM instance storage on each server using remote-exec. So create 3 servers and then mount the eNVM on each of the 3 servers
attempted to use null_resource but I am getting errors about 'resource depends on non-existent resource' or 'interpolation' errors
variable count {
default = 3
}
module "app-data-node" {
source = "some_git_source"
count = "${var.count}"
instance_size = "instance_data"
hostname_pattern = "app-data"
dns_domain = "${data.terraform_remote_state.network.dns_domain}"
key_name = "app-automation"
description = "Automation App Data Instance"
package_proxy = "${var.package_proxy}"
}
resource "null_resource" "mount_envm" {
# Only run this provisioner for app nodes
#count = "${var.count}"
depends_on = [
"null_resource.${module.app-data-node}"
]
connection {
host = "${aws_instance.i.*.private_ip[count.index]}"
user = "root"
private_key = "app-automation"
}
provisioner "remote-exec" {
inline = [
"sudo mkfs -t ext4 /dev/nvme0n1",
"sudo mkdir /data",
"sudo mount /dev/nvme0n1 /data"
]
}
}
3 EC2 instances each with eNVMs mounted on them.
You can use a null_resource to run the provisioner:
resource "null_resource" "provisioner" {
count = "${var.count}"
triggers {
master_id = "${element(aws_instance.my_instances.*.id, count.index)}"
}
connection {
#host = "${element(aws_instance.my_instances.*.private_ip, count.index)}"
host = "${element(aws_instance.my_instances.*.private_ip, count.index)}"
type = "ssh"
user = "..."
private_key = "..."
}
# set hostname
provisioner "remote-exec" {
inline = [
"sudo mkfs -t ext4 /dev/nvme0n1",
"sudo mkdir /data",
"sudo mount /dev/nvme0n1 /data"
]
}
}
This should do it for all instances at once as well.