I was able to manually create this device instance in OpenStack, and now I am trying to see how to make it work by terraform.
Anyway, this device instance need to do a hard reboot after the volume attachment, and Any cloud-config needs be down after rebooting. Here is the general sketch of my current main.tf file.
# Configure the OpenStack Provider
terraform {
required_providers {
openstack = {
source = "terraform-provider-openstack/openstack"
}
}
}
data "template_file" "user_data"{
template = file("./userdata.yaml")
}
# create an instance
resource "openstack_compute_instance_v2" "server" {
name = "Device_Instance"
image_id = "xxx..."
image_name = "device_vmdk_1"
flavor_name = "m1.tiny"
key_pair = "my-keypair"
region = "RegionOne"
config_drive = true
network {
name = "main_network"
}
}
resource "openstack_blockstorage_volume_v2" "volume_2" {
name = "device_vmdk_2"
size = 1
image_id = "xxx...."
}
resource "openstack_blockstorage_volume_v3" "volume_3" {
name = "device_vmdk_3"
size = 1
image_id = "xxx..."
}
resource "openstack_compute_volume_attach_v2" "va_1" {
instance_id = "${openstack_compute_instance_v2.server.id}"
volume_id = "${openstack_blockstorage_volume_v2.volume_2.id}"
}
resource "openstack_compute_volume_attach_v2" "va_2" {
instance_id = "${openstack_compute_instance_v2.server.id}"
volume_id = "${openstack_blockstorage_volume_v3.volume_3.id}"
}
resource "null_resource" "reboot_instance" {
provisioner "local-exec" {
on_failure = fail
interpreter = ["/bin/bash", "-c"]
command = <<EOT
openstack server reboot --hard Device_Instance --insecure
echo "................"
EOT
}
depends_on = [openstack_compute_volume_attach_v2.va_1, openstack_compute_volume_attach_v2.va_2]
}
resource "openstack_compute_instance_v2" "server_config" {
name = "Device_Instance"
user_data = data.template_file.user_data.rendered
depends_on = [null_resource.reboot_instance]
}
As of now, it was able to:
have the "Device-Cloud-Instance" generated.
have the "Device-Cloud-Instance" hard-rebooted.
But it fails after rooting. As you may find, I have added this section in the end, but it seems not working.
resource "openstack_compute_instance_v2" "server_config"{}
Any ideas how to make it work?
Thanks,
Jack
I am trying to create a new project and then a new Google Artifact Registry within the new project. Here are the Terraform resources:
resource "google_project" "my_project" {
name = "My Project Name"
project_id = "my-project-id-abc"
billing_account = "BILLING-ACCOUNT-ID"
}
resource "google_artifact_registry_repository" "my_ar" {
provider = google-beta
format = "DOCKER"
repository_id = "myreponame"
location = "europe-west1"
project = google_project.my_project.project_id
depends_on = [google_project_service.artifactregistry_googleapis_com]
}
resource "google_project_service" "artifactregistry_googleapis_com" {
project = google_project.my_project.project_id
service = "artifactregistry.googleapis.com"
}
This almost always fails on first the terraform apply with the following error message:
Error: Error creating Repository: googleapi: Error 403: Permission 'artifactregistry.repositories.create' denied on resource '//artifactregistry.googleapis.com/projects/my-project-id-abc/locations/europe-west1' (or it may not exist).
Details:
[
{
"#type": "type.googleapis.com/google.rpc.ErrorInfo",
"domain": "artifactregistry.googleapis.com",
"metadata": {
"permission": "artifactregistry.repositories.create",
"resource": "projects/my-project-id-abc/locations/europe-west1"
},
"reason": "IAM_PERMISSION_DENIED"
}
]
Running the same command immediately again always succeeds with the following message:
Terraform will perform the following actions:
# google_artifact_registry_repository.my_ar will be created
+ resource "google_artifact_registry_repository" "my_ar" {
+ create_time = (known after apply)
+ format = "DOCKER"
+ id = (known after apply)
+ location = "europe-west1"
+ name = (known after apply)
+ project = "my-project-id-abc"
+ repository_id = "myreponame"
+ update_time = (known after apply)
}
Plan: 1 to add, 0 to change, 0 to destroy.
google_artifact_registry_repository.my_ar: Creating...
google_artifact_registry_repository.my_ar: Still creating... [10s elapsed]
google_artifact_registry_repository.my_ar: Creation complete after 12s [id=projects/my-project-id-abc/locations/europe-west1/repositories/myreponame]
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
It the role of the depends_on = [google_project_service.artifactregistry_googleapis_com] attribute not support to wait for everything to be ready before creating the artifact repository?
Using the null_resource resource to delay things provided a temporary fix:
resource "google_artifact_registry_repository" "my_ar" {
project = google_project.my_project.project_id
provider = google-beta
format = "DOCKER"
repository_id = "myreponame"
location = "europe-west1"
depends_on = [null_resource.delay]
}
# in many scenarios the above artifact registries are created while the apis are not yet active
# this is a know issue: https://github.com/hashicorp/terraform-provider-google/issues/9902
# and this delay buys some time before creating the above repositories.
resource "null_resource" "delay" {
depends_on = [ google_project_service.artifactregistry_googleapis_com ]
provisioner "local-exec" {
command = "sleep 120"
}
triggers = {
project = google_project. my_project.id
}
}
In my Azure subscription, I'm trying to create an AKS cluster using Terraform.
My main.tf looks like this:
## Azure resource provider ##
provider "azurerm" {
version = "=1.36.1"
}
## Azure resource group for the kubernetes cluster ##
resource "azurerm_resource_group" "aks_demo" {
name = var.resource_group_name
location = var.location
}
## AKS kubernetes cluster ##
resource "azurerm_kubernetes_cluster" "aks_demo" {
name = var.cluster_name
resource_group_name = azurerm_resource_group.aks_demo.name
location = azurerm_resource_group.aks_demo.location
dns_prefix = var.dns_prefix
linux_profile {
admin_username = var.admin_username
## SSH key is generated using "tls_private_key" resource
ssh_key {
key_data = "${trimspace(tls_private_key.key.public_key_openssh)} ${var.admin_username}#azure.com"
}
}
agent_pool_profile {
name = "default"
count = var.agent_count
vm_size = "Standard_D2"
os_type = "Linux"
os_disk_size_gb = 30
}
service_principal {
client_id = var.client_id
client_secret = var.client_secret
}
tags = {
Environment = "Production"
}
}
## Private key for the kubernetes cluster ##
resource "tls_private_key" "key" {
algorithm = "RSA"
}
## Save the private key in the local workspace ##
resource "null_resource" "save-key" {
triggers = {
key = tls_private_key.key.private_key_pem
}
provisioner "local-exec" {
command = <<EOF
mkdir -p ${path.module}/.ssh
echo "${tls_private_key.key.private_key_pem}" > ${path.module}/.ssh/id_rsa
chmod 0600 ${path.module}/.ssh/id_rsa
EOF
}
}
## Outputs ##
# Example attributes available for output
output "id" {
value = "${azurerm_kubernetes_cluster.aks_demo.id}"
}
output "client_key" {
value = "${azurerm_kubernetes_cluster.aks_demo.kube_config.0.client_key}"
}
output "client_certificate" {
value = "${azurerm_kubernetes_cluster.aks_demo.kube_config.0.client_certificate}"
}
output "cluster_ca_certificate" {
value = "${azurerm_kubernetes_cluster.aks_demo.kube_config.0.cluster_ca_certificate}"
}
output "kube_config" {
value = azurerm_kubernetes_cluster.aks_demo.kube_config_raw
}
output "host" {
value = azurerm_kubernetes_cluster.aks_demo.kube_config.0.host
}
output "configure" {
value = <<CONFIGURE
Run the following commands to configure kubernetes client:
$ terraform output kube_config > ~/.kube/aksconfig
$ export KUBECONFIG=~/.kube/aksconfig
Test configuration using kubectl
$ kubectl get nodes
CONFIGURE
}
My variables.tf looks like this:
## Azure config variables ##
variable "client_id" {}
variable "client_secret" {}
variable location {
default = "Central US"
}
## Resource group variables ##
variable resource_group_name {
default = "aksdemo-rg"
}
## AKS kubernetes cluster variables ##
variable cluster_name {
default = "aksdemo1"
}
variable "vm_size" {
default = "Standard_A0"
}
variable "agent_count" {
default = 3
}
variable "dns_prefix" {
default = "aksdemo"
}
variable "admin_username" {
default = "demo"
}
When I run terraform apply, I get this error:
Error: Error creating Managed Kubernetes Cluster "aksdemo1" (Resource Group "aksdemo-rg"):
containerservice.ManagedClustersClient#CreateOrUpdate: Failure sending request: StatusCode=400 --
Original Error: Code="BadRequest"
Message="The VM size of AgentPoolProfile:default is not allowed in your subscription in location 'centralus'. The available VM sizes are Standard_A2,Standard_A2_v2,Standard_A2m_v2,Standard_A3,Standard_A4,Standard_A4_v2,Standard_A4m_v2,
Standard_A5,Standard_A6,Standard_A7,Standard_A8_v2,Standard_A8m_v2,Standard_B12ms,Standard_B16ms,Standard_B20ms,Standard_B2ms,Standard_B2s,Standard_B4ms,Standard_B8ms,Standard_D11_v2,Standard_D12_v2,
Standard_D13_v2,Standard_D14_v2,Standard_D15_v2,Standard_D16_v3,Standard_D16s_v3,Standard_D1_v2,Standard_D2_v2,Standard_D2_v3,Standard_D2s_v3,Standard_D32_v3,Standard_D32s_v3,Standard_D3_v2,Standard_D48_v3,
Standard_D48s_v3,Standard_D4_v2,Standard_D4_v3,Standard_D4s_v3,Standard_D5_v2,Standard_D64_v3,Standard_D64s_v3,Standard_D8_v3,Standard_D8s_v3,Standard_DS1,Standard_DS11,Standard_DS11_v2,Standard_DS12,Standard_DS12_v2,Standard_DS13,Standard_DS13-2_v2,Standard_DS13-4_v2,Standard_DS13_v2,Standard_DS14,Standard_DS14-4_v2,Standard_DS14-8_v2,Standard_DS14_v2,Standard_DS15_v2,Standard_DS1_v2,Standard_DS2,Standard_DS2_v2,Standard_DS3,Standard_DS3_v2,Standard_DS4,Standard_DS4_v2,Standard_DS5_v2,Standard_E16_v3,Standard_E16s_v3,Standard_E2_v3,Standard_E2s_v3,Standard_E32-16s_v3,Standard_E32-8s_v3,Standard_E32_v3,Standard_E32s_v3,Standard_E48_v3,Standard_E48s_v3,Standard_E4_v3,Standard_E4s_v3,Standard_E64-16s_v3,Standard_E64-32s_v3,Standard_E64_v3,Standard_E64i_v3,Standard_E64is_v3,Standard_E64s_v3,Standard_E8_v3,Standard_E8s_v3,Standard_F16,Standard_F16s,Standard_F16s_v2,Standard_F2,Standard_F2s,Standard_F2s_v2,Standard_F32s_v2,Standard_F4,Standard_F48s_v2,Standard_F4s,Standard_F4s_v2,Standard_F64s_v2,Standard_F72s_v2,Standard_F8,
Standard_F8s,Standard_F8s_v2
For more details, please visit https://aka.ms/cpu-quota"
This is confusing to me, as there is clearly a variable named vm_size
What can I change in order for this to work?
As I see from the code you provided and the error you got, you made the mistake in the code.
What the code you made:
agent_pool_profile {
name = "default"
count = var.agent_count
vm_size = "Standard_D2"
os_type = "Linux"
os_disk_size_gb = 30
}
It should be like this when you use the variable for the VM size:
agent_pool_profile {
name = "default"
count = var.agent_count
vm_size = var.vm_size
os_type = "Linux"
os_disk_size_gb = 30
}
And the VM size should be an appropriate one on yourself and for the requirements. For example, just like it shows in the Terraform example.
The error message is telling you that you're trying to use a VM size, or VM type if you will that's not available for your subscription in that location, it's also giving you all the VM sizes you can choose from.
Note that you have probably copy pasted this:
agent_pool_profile {
name = "default"
count = var.agent_count
vm_size = "Standard_D2"
os_type = "Linux"
os_disk_size_gb = 30
}
The VM size is hardcoded there, so you're default Standard_A0value is not being picked up.
You have more than one way to debug here, first I'd start by making sure the right value is being used, and second change the VM type to see if that works.
I'm struggling to get the password from a couple of new ec2 instances when using terraform. Been reading up through a couple of posts and thought i had it but not getting anywhere.
Here's my config:
resource "aws_instance" "example" {
ami = "ami-06f9d25508c9681c3"
count = "2"
instance_type = "t2.small"
key_name = "mykey"
vpc_security_group_ids =["sg-98d190fc","sg-0399f246d12812edb"]
get_password_data = "true"
}
output "public_ip" {
value = "${aws_instance.example.*.public_ip}"
}
output "public_dns" {
value = "${aws_instance.example.*.public_dns}"
}
output "Administrator_Password" {
value = "${rsadecrypt(aws_instance.example.*.password_data,
file("mykey.pem"))}"
}
Managed to clear up all the syntax errors but now when running get the following error:
PS C:\tf> terraform apply
aws_instance.example[0]: Refreshing state... (ID: i-0e087e3610a8ff56d)
aws_instance.example[1]: Refreshing state... (ID: i-09557bc1e0cb09c67)
Error: Error refreshing state: 1 error(s) occurred:
* output.Administrator_Password: At column 3, line 1: rsadecrypt: argument 1
should be type string, got type list in:
${rsadecrypt(aws_instance.example.*.password_data, file("mykey.pem"))}
This error is returned because aws_instance.example.*.password_data is a list of the password_data results from each of the EC2 instances. Each one must be decrypted separately with rsadecrypt.
To do this in Terraform v0.11 requires using null_resource as a workaround to achieve a "for each" operation:
resource "aws_instance" "example" {
count = 2
ami = "ami-06f9d25508c9681c3"
instance_type = "t2.small"
key_name = "mykey"
vpc_security_group_ids = ["sg-98d190fc","sg-0399f246d12812edb"]
get_password_data = true
}
resource "null_resource" "example" {
count = 2
triggers = {
password = "${rsadecrypt(aws_instance.example.*.password_data[count.index], file("mykey.pem"))}"
}
}
output "Administrator_Password" {
value = "${null_resource.example.*.triggers.password}"
}
From Terraform v0.12.0 onwards, this can be simplified using the new for expression construct:
resource "aws_instance" "example" {
count = 2
ami = "ami-06f9d25508c9681c3"
instance_type = "t2.small"
key_name = "mykey"
vpc_security_group_ids = ["sg-98d190fc","sg-0399f246d12812edb"]
get_password_data = true
}
output "Administrator_Password" {
value = [
for i in aws_instance.example : rsadecrypt(i.password_data, file("mykey.pem"))
]
}
I have a question relating to AWS RDS cluster and instance creation.
Environment
We recently experimented with:
Terraform v0.11.11
provider.aws v1.41.0
Background
Creating some AWS RDS databases. Our mission was that in some environment (e.g. staging) we may run fewer instances than in others (e.g. production.). With this in mind and not wanting to have totally different terraform files per environment we instead decided to specify the database resources just once and use a variable for the number of instances which is set in our staging.tf and production.tf files respectively for the number of instances.
Potentially one more "quirk" of our setup, is that the VPC in which the subnets exist is not defined in terraform, the VPC already existed via manual creation in the AWS console, so this is provided as a data provider and the subnets for the RDS are specific in terraform - but again this is dynamic in the sense that in some environments we might have 3 subnets (1 in each AZ), whereas in others perhaps we have only 2 subnets. Again to achieve this we used iteration as shown below:
Structure
|-/environments
-/staging
-staging.tf
-/production
-production.tf
|- /resources
- database.tf
Example Environment Variables File
dev.tf
terraform {
terraform {
backend "s3" {
bucket = "my-bucket-dev"
key = "terraform"
region = "eu-west-1"
encrypt = "true"
acl = "private"
dynamodb_table = "terraform-state-locking"
}
version = "~> 0.11.8"
}
provider "aws" {
access_key = "${var.access_key}"
secret_key = "${var.secret_key}"
region = "${var.region}"
version = "~> 1.33"
allowed_account_ids = ["XXX"]
}
module "main" {
source = "../../resources"
vpc_name = "test"
test_db_name = "terraform-test-db-dev"
test_db_instance_count = 1
test_db_backup_retention_period = 7
test_db_backup_window = "00:57-01:27"
test_db_maintenance_window = "tue:04:40-tue:05:10"
test_db_subnet_count = 2
test_db_subnet_cidr_blocks = ["10.2.4.0/24", "10.2.5.0/24"]
}
We came to this module based structure for environment isolation mainly due to these discussions:
https://github.com/hashicorp/terraform/issues/18632#issuecomment-412247266
https://github.com/hashicorp/terraform/issues/13700
https://www.terraform.io/docs/state/workspaces.html#when-to-use-multiple-workspaces
Our Issue
Initial resource creation works fine, our subnets are created, the database cluster starts up.
Our issues start the next time we subsequently run a terraform plan or terraform apply (with no changes to the files), at which point we see interesting things like:
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
-/+ destroy and then create replacement
Terraform will perform the following actions:
module.main.aws_rds_cluster.test_db (new resource required)
id: "terraform-test-db-dev" => (forces new resource)
availability_zones.#: "3" => "1" (forces new resource)
availability_zones.1924028850: "eu-west-1b" => "" (forces new resource)
availability_zones.3953592328: "eu-west-1a" => "eu-west-1a"
availability_zones.94988580: "eu-west-1c" => "" (forces new resource)
and
module.main.aws_rds_cluster_instance.test_db (new resource required)
id: "terraform-test-db-dev" => (forces new resource)
cluster_identifier: "terraform-test-db-dev" => "${aws_rds_cluster.test_db.id}" (forces new resource)
Something about the way we are approaching this appears to be causing terraform to believe that the resource has changed to such an extent that it must destroy the existing resource and create a brand new one.
Config
variable "aws_availability_zones" {
description = "Run the EC2 Instances in these Availability Zones"
type = "list"
default = ["eu-west-1a", "eu-west-1b", "eu-west-1c"]
}
variable "test_db_name" {
description = "Name of the RDS instance, must be unique per region and is provided by the module config"
}
variable "test_db_subnet_count" {
description = "Number of subnets to create, is provided by the module config"
}
resource "aws_security_group" "test_db_service" {
name = "${var.test_db_service_user_name}"
vpc_id = "${data.aws_vpc.vpc.id}"
}
resource "aws_security_group" "test_db" {
name = "${var.test_db_name}"
vpc_id = "${data.aws_vpc.vpc.id}"
}
resource "aws_security_group_rule" "test_db_ingress_app_server" {
security_group_id = "${aws_security_group.test_db.id}"
...
source_security_group_id = "${aws_security_group.test_db_service.id}"
}
variable "test_db_subnet_cidr_blocks" {
description = "Cidr block allocated to the subnets"
type = "list"
}
resource "aws_subnet" "test_db" {
count = "${var.test_db_subnet_count}"
vpc_id = "${data.aws_vpc.vpc.id}"
cidr_block = "${element(var.test_db_subnet_cidr_blocks, count.index)}"
availability_zone = "${element(var.aws_availability_zones, count.index)}"
}
resource "aws_db_subnet_group" "test_db" {
name = "${var.test_db_name}"
subnet_ids = ["${aws_subnet.test_db.*.id}"]
}
variable "test_db_backup_retention_period" {
description = "Number of days to keep the backup, is provided by the module config"
}
variable "test_db_backup_window" {
description = "Window during which the backup is done, is provided by the module config"
}
variable "test_db_maintenance_window" {
description = "Window during which the maintenance is done, is provided by the module config"
}
data "aws_secretsmanager_secret" "test_db_master_password" {
name = "terraform/db/test-db/root-password"
}
data "aws_secretsmanager_secret_version" "test_db_master_password" {
secret_id = "${data.aws_secretsmanager_secret.test_db_master_password.id}"
}
data "aws_iam_role" "rds-monitoring-role" {
name = "rds-monitoring-role"
}
resource "aws_rds_cluster" "test_db" {
cluster_identifier = "${var.test_db_name}"
engine = "aurora-mysql"
engine_version = "5.7.12"
# can only request to deploy in AZ's where there is a subnet in the subnet group.
availability_zones = "${slice(var.aws_availability_zones, 0, var.test_db_instance_count)}"
database_name = "${var.test_db_schema_name}"
master_username = "root"
master_password = "${data.aws_secretsmanager_secret_version.test_db_master_password.secret_string}"
preferred_backup_window = "${var.test_db_backup_window}"
preferred_maintenance_window = "${var.test_db_maintenance_window}"
backup_retention_period = "${var.test_db_backup_retention_period}"
db_subnet_group_name = "${aws_db_subnet_group.test_db.name}"
storage_encrypted = true
kms_key_id = "${data.aws_kms_key.kms_rds_key.arn}"
deletion_protection = true
enabled_cloudwatch_logs_exports = ["audit", "error", "general", "slowquery"]
vpc_security_group_ids = ["${aws_security_group.test_db.id}"]
final_snapshot_identifier = "test-db-final-snapshot"
}
variable "test_db_instance_count" {
description = "Number of instances to create, is provided by the module config"
}
resource "aws_rds_cluster_instance" "test_db" {
count = "${var.test_db_instance_count}"
identifier = "${var.test_db_name}"
cluster_identifier = "${aws_rds_cluster.test_db.id}"
availability_zone = "${element(var.aws_availability_zones, count.index)}"
instance_class = "db.t2.small"
db_subnet_group_name = "${aws_db_subnet_group.test_db.name}"
monitoring_interval = 60
engine = "aurora-mysql"
engine_version = "5.7.12"
monitoring_role_arn = "${data.aws_iam_role.rds-monitoring-role.arn}"
tags {
Name = "test_db-${count.index}"
}
}
My question is, is there a way to achieve this so that terraform would not try to recreate the resource (e.g. ensure that the availability zones of the cluster and ID of the instance do not change each time we run terraform.
Turns out that simply by just removing the explicit availability zones definitions from the aws_rds_cluster and aws_rds_cluster_instance then this issue goes away and everything so far appears to work as expected. See also https://github.com/terraform-providers/terraform-provider-aws/issues/7307#issuecomment-457441633