I am trying to learn terraform on OCI, I have written a small code to in my terraform-code.tf file to create a block instance, however when I run the terraform plan I get the following error.
data "oci_identity_availability_domain" "ad" {
compartment_id = "var.tenancy_ocid"
}
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
data.oci_identity_availability_domain.ad: Refreshing state...
Error: Get https://identity.var.region.oraclecloud.com/20160918/availabilityDomains?compartmentId=ocid1.tenancy.oc1..aaaaaaaa35fzgotfw445uiswdvjcxnxitafa4scy4dmcuifrvvzkxylqga3q: dial tcp: lookup identity.var.region.oraclecloud.com: no such host
on terraform-code.tf line 46, in data "oci_identity_availability_domain" "ad":
46: data "oci_identity_availability_domain" "ad" {
I tried to ping identity.var.region.oraclecloud.com from my windows machine but no luck
ping identity.var.region.oraclecloud.com
Ping request could not find host identity.var.region.oraclecloud.com. Please check the name and try again.
I believe this is an issue with the proxy where for some reason I am unable to reach
identity.var.region.oraclecloud.com
I found a similar article on github : https://github.com/terraform-providers/terraform-provider-oci/issues/960
Can anyone help me to resolve this issue ?
var.region is a variable and should be substitued. It's normal you can't reach https://identity.var.region.oraclecloud.com as it doesn't exist. Here is a list of the existing regions.
A valide url would be for instance https://identity.us-ashburn-1.oraclecloud.com
To answer my own question, the ping test to identity.var.region.oraclecloud.com does not matter.
If you receive the below error most probably you are not passing your region ocid correctly in the required variables. To troubleshoot you can replace the variables with the actual OCID in double quotes as a string ""
Error: Get https://identity.var.region.oraclecloud.com/20160918/availabilityDomains?compartmentId=ocid1.tenancy.oc1..aaaaaaaa35fzgotfw445uiswdvjcxnxitafa4scy4dmcuifrvvzkxylqga3q: dial tcp: lookup identity.var.region.oraclecloud.com: no such host
on terraform-code.tf line 46, in data "oci_identity_availability_domain" "ad":
46: data "oci_identity_availability_domain" "ad" {
For me the issue was,
I was passing the variable information incorrectly.
With TF 0.11 variable information is set like
tenancy_ocid = "${var.tenancy_ocid}"
With TF 0.13 variable information is set like
tenancy_ocid = "${var.tenancy_ocid}" ( the old way would still work but you will receive a warning)
Or for troubleshooting you can simply use
tenancy_ocid = ""
I have just started to learn terraform with OCI and there are not many helpful posts around.
Related
Context: I'm creating a new TF provider.
TF official docs say that
When you create something in Terraform but delete it manually, Terraform should gracefully handle it. If the API returns an error when the resource doesn't exist, the read function should check to see if the resource is available first. If the resource isn't available, the function should set the ID to an empty string so Terraform "destroys" the resource in state. The following code snippet is an example of how this can be implemented; you do not need to add this to your configuration for this tutorial.
if resourceDoesntExist {
d.SetID("")
return
}
It's pretty clear when resourceDoesntExist := response.code == 404 but what about the case where the resource is in DELETING state (which means that the resource is going to be removed in like 30 minutes and at that point GET request will start returning 404).
Shall it be treated as 404 too? What about the corresponding data source, shall it return an error?
Recently upgraded my Terraform project to AWS provider 3.74.0 and TF 1.1.4 (from much older versions).
I'm suddenly getting this autoscaling schedule reporting external changes:
resource "aws_autoscaling_schedule" "api-svc-tst-down-schedule" {
scheduled_action_name = "api-svc-tst-down-schedule"
min_size = 0
max_size = 1
desired_capacity = 0
// Minute Hour DayOfMonth Month DayOfWeek
recurrence = "0 13 * * *"
autoscaling_group_name = aws_autoscaling_group.api-svc-tst-asg.name
lifecycle {
ignore_changes = [start_time]
}
}
The plan command is now reporting:
Note: Objects have changed outside of Terraform
Terraform detected the following changes made outside of Terraform since the
last "terraform apply":
# aws_autoscaling_schedule.api-svc-tst-down-schedule has changed
~ resource "aws_autoscaling_schedule" "api-svc-tst-down-schedule" {
id = "api-svc-tst-down-schedule"
~ start_time = "2022-01-31T13:00:00Z" -> "2022-02-01T13:00:00Z"
# (7 unchanged attributes hidden)
}
If I apply the plan, it doesn't appear that TF changes the ASG (I'm assuming it just updates its state file) and the notification goes away until the next day.
I note that the AWS console does show that the Scheduled action has a Start time, which seems to be being set by AWS.
I tried adding start_time to ignored_changes but it didn't seem to make a difference, still reported as externally changed.
Is this a known issue with Terraform (I'm not seeing anything via googling)?
How can I prevent TF from being marked as externally changed?
Edit: I also tried setting the start_time attribute as suggested in the comments. But the detected changes warning came back the next day.
Edit 2: I also tried deleting and re-adding the resource via Terraform, but it still gets marked as changed the next day.
This undesirable behavior was an intentional change introduced in Terraform version 0.15.4.
It cannot currently be avoided. The only workaround is that all team members (and tooling) must be educated to ignore "expected drift".
Note that this "expected drift" behavior is not limited to just aws_autoscaling_schedule resources, or even just the AWS provider. The issue happens on many different platforms/types for any resource where the cloud vendor updates the attribute after the resource is created.
Many resources will report drift immediately after being created - often you can get rid of the report by immediately doing an apply or refresh to update the TF state and as long as AWS doesn't make changes to those attributes, you won't see the resource reported as changed again.
Other resource attributes (like aws_autoscaling_schedule.start_time) get updated by the cloud vendor regularly. These types of resources will intermittently report "Objects have changed outside of Terraform", whenever you run plan.
There is a locked open issue to track: https://github.com/hashicorp/terraform/issues/28803.
Note that the issue is locked because Hashicorp got tired of people telling them how negatively this affects their teams.
I have a very frustrating Terraform issue, I made some changes to my terraform script which failed when I applied the plan. I've gone through a bunch of machinations and probably made the situation worse as I ended up manually deleting a bunch of AWS resources in trying to resolve this.
So now I am unable to use Terraform at all (refresh, plan, destroy) all get the same error.
The Situation
I have a list of Fargate services, and a set of maps which correlate different features of the fargate services such as the "Target Group" for the load balancer (I've provided some code below). The problem appears to be that Terraform is not picking up that these resources have been manually deleted or is somehow getting confused because they don't exist. At this point if I run a refresh, plan or destroy I get an error stating that a specific list is empty, even though it isn't (or should not be).
In the failed run I added a new service to the list below along with a new url (see code below)
Objective
At this point I would settle for destroying the entire environment (its my dev environment), however; ideally I want to just get the system working such that Terraform will detect the changes and work properly.
Terraform Script is Valid
I have reverted my Terraform scripts back to the last known good version. I have run the good version against our staging environment and it works fine.
Configuration Info
MacOS Mojave 10.14.6 (18G103)
Terraform v0.12.24.
provider.archive v1.3.0
provider.aws v2.57.0
provider.random v2.2.1
provider.template v2.1.2
The Terraform state file is being stored in a S3 bucket, and terraform init --reconfigure has been called.
What I've done
I was originally getting a similar error but it was in a different location, after many hours Googling and trying stuff (which I didn't write down) I decided to manually remove the AWS resources associated with the problematic code (the ALB, Target Groups, security groups)
Example Terraform Script
Unfortunately I can't post the actual script as it is private, but I've posted what I believe is the pertinent parts but have redacted some info. The reason I mention this is that any syntax type error you might see would be caused by this redaction, as I stated above the script works fine when run in our staging environment.
globalvars.tf
In the root directory. In the case of the failed Terraform run I added a new name to the service_names (edd = "edd") list (I added as the first element). In the service_name_map_2_url I added the new entry (edd = "edd") as the last entry. I'm not sure if the fact that I added these elements in different 'order' is the problem, although it really shouldn't since I access the map via the name and not by index
variable "service_names" {
type = list(string)
description = "This is a list/array of the images/services for the cluster"
default = [
"alert",
"alert-config"
]
}
variable service_name_map_2_url {
type = map(string)
description = "This map contains the base URL used for the service"
default = {
alert = "alert"
alert-config = "alert-config"
}
}
alb.tf
In modules/alb. In this module we create an ALB and then a target group for each service, which looks like this. The items from globalvars.tf are passed into this script
locals {
numberOfServices = length(var.service_names)
}
resource "aws_alb" "orchestration_alb" {
name = "orchestration-alb"
subnets = var.public_subnet_ids
security_groups = [var.alb_sg_id]
tags = {
environment = var.environment
group = var.tag_group_name
app = var.tag_app_name
contact = var.tag_contact_email
}
}
resource "aws_alb_target_group" "orchestration_tg" {
count = local.numberOfServices
name = "${var.service_names[count.index]}-tg"
port = 80
protocol = "HTTP"
vpc_id = var.vpc_id
target_type = "ip"
deregistration_delay = 60
tags = {
environment = var.environment
group = var.tag_group_name
app = var.tag_app_name
contact = var.tag_contact_email
}
health_check {
path = "/${var.service_name_map_2_url[var.service_names[count.index]]}/health"
port = var.app_port
protocol = "HTTP"
healthy_threshold = 2
unhealthy_threshold = 5
interval = 30
timeout = 5
matcher = "200-308"
}
}
output.tf
This is the output of the alb.tf, other things are outputted but this is the one that matters for this issue
output "target_group_arn_suffix" {
value = aws_alb_target_group.orchestration_tg.*.arn_suffix
}
cloudwatch.tf
In modules/cloudwatch. I attempt to create a dashboard
data "template_file" "Dashboard" {
template = file("${path.module}/dashboard.json.template")
vars = {
...
alert-tg = var.target_group_arn_suffix[0]
alert-config-tg = var.target_group_arn_suffix[1]
edd-cluster-name = var.ecs_cluster_name
alb-arn-suffix = var.alb-arn-suffix
}
}
Error
When I run terraform refresh (or plan or destroy) I get the following error (I get the same error for alert-config as well)
Error: Invalid index
on modules/cloudwatch/cloudwatch.tf line 146, in data "template_file" "Dashboard":
146: alert-tg = var.target_group_arn_suffix[0]
|----------------
| var.target_group_arn_suffix is empty list of string
The given key does not identify an element in this collection value.
AWS Environment
I have manually deleted the ALB. Dashboard and all Target Groups. I would expect (and this has worked in the past) that Terraform would detect this and update its state file appropriately such that when running a plan it would know it has to create the ALB and target groups.
Thank you
Terraform trusts its state as the single source of truth. Using Terraform in the presence of manual change is possible, but problematic.
If you manually remove infrastructure, you need to run terraform state rm [resource path] on the manually removed resource.
Gruntwork has what they call The Golden Rule of Terraform:
The master branch of the live repository should be a 1:1 representation of what’s actually deployed in production.
I am new to Terraform and trying to understand data sources. I have read the documentation and this StackOverflow post, but I'm still unclear about the use cases of data source.
I have the following block of code:
resource "azurerm_resource_group" "rg" {
name = "example-resource-group"
location = "West US 2"
}
data "azurerm_resource_group" "test" {
name = "example-resource-group"
}
But I get a 404 error:
data.azurerm_resource_group.test: data.azurerm_resource_group.test: resources.GroupsClient#Get: Failure responding to request:
StatusCode=404 -- Original Error: autorest/azure: Service returned an
error. Status=404 Code="ResourceGroupNotFound" Message="Resource group
'example-resource-group' could not be found."
I don't understand why the resource group is not found. Also, I am unclear about the difference between data and variable and when should I use which.
Thanks
I have provided a detailed explanation of what a data source is in this SO answer. To summarize:
Data sources provide dynamic information about entities that are not managed by the current Terraform configuration
Variables provide static information
Your block of code doesn't work because the resource your data source is referencing hasn't been created yet. During the planning phase, Terraform will try to find a resource group named example-resource-group, but it won't find it, and so it aborts the whole run. The ordering of the blocks makes no difference to the order they are applied.
If you remove the data block, run terraform apply, and then add the data block back in, it should work. However, data sources are used to retrieve data about entities that are not managed by your Terraform configuration. In your case, you don't need the data.azurerm_resource_group.test data source, you can simply use the exported attributes from the resource. In the case of azurerm_resource_group, this is a single id attribute.
Think of a data source as a value you want to read from somewhere else.
A variable is something you define when you run the code.
When you use the data source for azurerm_resource_group terraform will search for an existing resource that has the name you defined in your data source block.
Example
data "azurerm_resource_group" "test" {
name = "example-resource-group"
}
Quoting #ydaetskcoR from the comment below about 404 error:
It's 404ing because the data source is running before the resource
creates the thing you are looking for. You would use a data source
when the resource has already been created previously, not in the same
run as the resource you are creating.
I have a remote state attribute called subnets which is stored in: data.terraform_remote_state.alb.subnets
Depending on what I'm deploying, this attribute either exists or doesn't exist.
When I try to create an ECS cluster, it requires an input of the subnet groups in which I would like to either use:
data.terraform_remote_state.alb.subnets
or
var.vpc_subnets (the subnets of the VPC)
Unfortunately, because of the way the interpolation works, it needed to be hacked together:
"${split(",", length(var.vpc_subnets) == 0 ? join(",",data.terraform_remote_state.alb.subnets) : join(",",var.vpc_subnets))}"
(Refering to: https://github.com/hashicorp/terraform/issues/12453)
However, because Terraform does not seem to 'lazily' evaluate ternary operators, it throws me the following error even if var.vpc_subnets is NOT zero:
Resource 'data.terraform_remote_state.alb' does not have attribute 'subnets' for variable 'data.terraform_remote_state.alb.subnets'
How can I properly handle remote state resources that could be undefined?
EDIT: Typo: Subnet->Subnets
Managed to figure it out.
When using Terraform Remote State, you have the ability to set a default: https://www.terraform.io/docs/providers/terraform/d/remote_state.html
This works in my situation when my data "terraform_remote_state.alb.subnets does not return a value. I can preset the variable to be "" and use locals to do a check for this variable.
Will it be subnet or subnets?
Suppose you have below data source:
data "terraform_remote_state" "alb" {
backend = "s3"
config {
name = "alb"
}
}
You need check the remote state attribute have any outputs with name subnet or not. Or the key name is subnets, you need confirm by yourself.