I am trying to create regional managed instance group using terraform but facing the following error message:
1 error occurred:
* module.gcp-lb.google_compute_backend_service.default: 1 error occurred:
* google_compute_backend_service.default: Invalid value for group: A zone must be specified
I am not sure why resource google_compute_backend_service is asking for the zone if it is a regional managed instance group. Also zone seems to be not supported for this resource. I am following tasks:
create instance template with google_compute_instance_template
consume template in google_compute_region_instance_group_manager
passing the instance_group to backend in google_compute_backend_service with below syntax:
backend = ["${var.backends}"]
I am passing the backends from the module with below synatx:
backends = {
"0" = [
{ group = "${module.gcp-mig.instance_group}" }
],
}
backend_params = [
"/,http,80,10"
]
}
I am using terraform version: 0.11.13. Any kind of help/pointer is highly appreciated.
Thanks,
Sanjiv
It seems issue is with 2.5.0 google provider. It is working fine with google provider version 2.3.0
Related
Trying to migrate event monitors according to migration steps
resource "datadog_monitor" "cache_event" {
name = "${var.cache_replication_group} :: Event"
type = "event-v2 alert"
message = join(" ", ["Cache event occured in ${var.cache_replication_group}.", join(" ", var.recipients.general)])
query = "events(\"source:elasticache tags:(replication_group:${var.cache_replication_group}) NOT(Finished)\").rollup(\"count\").last(\"15m\") > 0"
require_full_window = false
tags = concat(var.tags, ["cache:${var.cache_replication_group}", "monitor:stability"])
}
But when trying to apply changes Terraform returns the error Error: error creating monitor: event-v2 alert is not a valid MonitorType, but according to their documentation event-v2 alert is a valid type. DataDog provider version at the moment is 3.10.0, hence it should support this type
My bad. After changing the provider version, I didn't reinitialise the state file.
Edit:
Before monitor migration, I used an older DataDog provider version. After adjusting query to be complient with a new event monitor format, I bumped DataDog provider version to the one that supports new v2 monitors, but instead of deleting .terraform folder and triggering init Terraform command, I just went straight with apply. Instead I should have initialized it first (init).
I am trying to create a AWS CodeBuild using Terraform.
resource "aws_codebuild_project" "cicd_codebuild" {
name = "cicd-${var.profile}-build"
description = "cicd ${var.profile} CodeBuild"
service_role = "${aws_iam_role.cicd_role.arn}"
source {
type = "GITHUB_ENTERPRISE"
location = "https://git.xxx.com/yyy/zzz.git"
git_clone_depth = 0
buildspec = "NO_SOURCE"
}
environment {
compute_type = "BUILD_GENERAL1_MEDIUM"
image = "aws/codebuild/windows-base:2019-1.0"
type = "WINDOWS_SERVER_2019_CONTAINER"
image_pull_credentials_type = "CODEBUILD"
}
artifacts {
type = "NO_ARTIFACTS"
}
}
Upon terraform apply I get error:
Error: aws_codebuild_project.cicd_codebuild: expected environment.0.type to be one of [LINUX_CONTAINER LINUX_GPU_CONTAINER WINDOWS_CONTAINER ARM_CONTAINER], got WINDOWS_SERVER_2019_CONTAINER
And when I change value of environment.0.type = "WINDOWS_CONTAINER" I get below error:
Error: Error applying plan:
1 error occurred:
* aws_codebuild_project.cicd_codebuild: 1 error occurred:
* aws_codebuild_project.cicd_codebuild: Error creating CodeBuild project: InvalidInputException: The environment type WINDOWS_CONTAINER is deprecated for new projects or existing project environment updates. Please consider using Windows Server 2019 instead.
I found on GitHub that this issue has been addressed in next versions. So, I know upgrading provider version can solve this but do we have any workaround to fix this issue in same version of Terraform and Provider.
Thanks.
Terraform has plan time validation on many resource parameters that allows for catching where you are passing an invalid parameter before you get to the point of trying to apply it.
Normally this is beneficial but if you are not able to keep up to date with the provider versions it means that that list of allowed values can get out of date with what is actually allowed by the backing service the provider is talking to.
In this specific case a pull request added the WINDOWS_SERVER_2019_CONTAINER as a plan time validation option after AWS added that functionality in July 2020.
Unfortunately for you, this work was merged and released as part of the v3.20.0 release of the AWS provider and the v3 releases only support Terraform 0.12 and up:
BREAKING CHANGES
provider: New versions of the provider can only be automatically installed on Terraform 0.12 and later (#14143)
If you want to be able to use Windows containers in CodeBuild you either need to upgrade to a more recent version of Terraform and the AWS provider or you need to use a different tool for creating the CodeBuild project.
One potential workaround here is to use CloudFormation to create the CodeBuild project which you could run via Terraform using the aws_cloudformation_stack resource.
Did anyone experienced issues with Terraform being throttled when using it with AWS Route53 records and being VERY slow?
I have enabled DEBUG mode and getting this:
2018-11-30T14:35:08.467Z [DEBUG] plugin.terraform-provider-aws_v1.36.0_x4: 2018/11/30 14:35:08 [DEBUG] [aws-sdk-go] <?xml version="1.0"?>
2018-11-30T14:35:08.467Z [DEBUG] plugin.terraform-provider aws_v1.36.0_x4: <ErrorResponse xmlns="https://route53.amazonaws.com/doc/2013-04-01/"><Error><Type>Sender</Type><Code>Throttling</Code><Message>Rate exceeded</Message></Error><RequestId>REQUEST_ID</RequestId></ErrorResponse>
2018-11-30T14:35:08.518Z [DEBUG] plugin.terraform-provider-aws_v1.36.0_x4: 2018/11/30 14:35:08 [DEBUG] [aws-sdk-go] DEBUG: Validate Response route53/ListResourceRecordSets failed, will retry, error Throttling: Rate exceeded
Terraform takes >1h just to do simple Plan, something which normally takes <5 mins.
My infrastructure is organized like this:
alb.tf:
module "ALB"
{ source = "modules/alb" }
modules/alb/alb.tf:
resource "aws_alb" "ALB"
{ name = "alb"
subnets = var.subnets ...
}
modules/alb/dns.tf
resource "aws_route53_record" "r53" {
count = "${length(var.cnames_generic)}"
zone_id = "HOSTED_ZONE_ID"
name = "${element(var.cnames_generic_dns, count.index)}.${var.environment}.${var.domain}"
type = "A"
alias {
name = "dualstack.${aws_alb.ALB.dns_name}"
zone_id = "${aws_alb.ALB.zone_id}"
evaluate_target_health = false
}
}
modules/alb/variables.tf:
variable "cnames_generic_dns" {
type = "list"
default = [
"hostname1",
"hostname2",
"hostname3",
"hostname4",
"hostname5",
"hostname6",
"hostname7",
...
"hostname25"
]
}
So I am using modules to configure Terraform, and inside modules there are resources (ALB, DNS..).
However, looks like Terraform is describing every single DNS Resource (CNAME and A records, which I have ~1000) in a HostedZone which is causing it to Throttle?
Terraform v0.10.7
Terraform AWS provider version = "~> 1.36.0"
that's a lot of DNS records! And partly the reason why the AWS API is throttling you.
First, I'd recommend upgrading your AWS provider. v1.36 is fairly old and there have been more than a few bug fixes since.
(Next, but not absolutely necessary, is to use TF v0.11.x if possible.)
In your AWS Provider block, increase max_retries to at least 10 and experiment with higher values.
Then, use Terraform's --parallelism flag to limit TF's concurrency rate. Try setting that to 5 for starters.
Last, enable Terraform's debug mode to see if it gives you any more useful info.
Hope this helps!
The problem is solved by performing the following actions:
since we re-structured DNS records by adding one resource and then variables / iterate through them, this probably caused Terraform to query constantly all DNS records
we decided to leave Terraform to finish refresh (took 4h and lots of throttling)
manually deleted DNS records from R53 for the Workspace which we were doing this
commenting out Terraform DNS resources so let it also delete from state files
uncommenting Terraform DNS and re-run it again so it created them again
run Terraform plan went fine again
Looks like throttling with Terraform AWS Route53 is completely resolved after upgrading to newer AWS provider. We have updated TF AWS provider to 1.54.0 like this in our init.tf :
version = "~> 1.54.0"
Here are more details about the issue and suggestions from Hashicorp engineers:
https://github.com/terraform-providers/terraform-provider-aws/issues/7056
I am new in terraform . I have a issue which I am facing when I am launching a simple EMR cluster in private subnet
It fails with the below error message :
aws_emr_cluster.emr-test-cluster: [WARN] Error waiting for EMR Cluster state to be "WAITING" or "RUNNING": TERMINATED_WITH_ERRORS: VALIDATION_ERROR: You must also specify a ServiceAccessSecurityGroup if you use custom security groups when creating a cluster in a private subnet.
I did check the github seems like the fixed it for the issue opened . But I am using the latest version of terraform (0.11.7)
Below are the github links for for the issue reported in Github
https://github.com/hashicorp/terraform/issues/9518
https://github.com/hashicorp/terraform/pull/9600
Any suggestions on how to fix this will be really helpful
Thank you
The issue is fixed in Git because it was raised to show an error which is supposed to ask for service_access_security_group while using the emr_managed_master_security_group and emr_managed_slave_security_group.
So, you would require mentioning service_access_security_group parameter in your EMR resource.
Thanks.
As we know that to put emr components in private subnet, you have to following
these security groups
https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-man-sec-groups.html#emr-sg-elasticmapreduce-sa-private
Then in terraform you have to resolve cyclic dependency between security groups from this link and put following configuration as shown below
ec2_attributes {
subnet_id = element(var.subnet_ids, count.index)
key_name = "${var.ssh_key_id}"
emr_managed_master_security_group = aws_security_group.EmrManagedMasterSecurityGroup.id
emr_managed_slave_security_group = aws_security_group.EmrManagedSlaveSecurityGroup.id
service_access_security_group = aws_security_group.ServiceAccessSecurityGroup.id
#additional_master_security_groups = aws_security_group.allow_ssh.id
instance_profile = aws_iam_instance_profile.example_ec2_profile.arn
}
source:- https://github.com/hashicorp/terraform/pull/9600
I am trying to declare the following Terraform provider:
provider "mysql" {
endpoint = "${aws_db_instance.main.endpoint}:3306"
username = "root"
password = "root"
}
I get the following error:
Error refreshing state: 1 error(s) occurred:
* dial tcp: lookup ${aws_db_instance.main.endpoint}: invalid domain name
It seems that Terraform is not performing interpolation on my endpoint string, yet I don't see anything in the documentation about this -- what gives?
Yes, it does. There's an example in the docs at https://www.terraform.io/docs/providers/mysql/
# Configure the MySQL provider based on the outcome of
# creating the aws_db_instance.
provider "mysql" {
endpoint = "${aws_db_instance.default.endpoint}"
username = "${aws_db_instance.default.username}"
password = "${aws_db_instance.default.password}"
}
I ran into a similar set of error messages ("connect failed," "invalid domain lookup") and looked into this a bit. I hope this helps you or someone else working across cloud and database providers in Terraform.
This seems to come down to the MySQL provider attempting to establish a database connection as soon as it's initialized, which could be a problem if you're trying to build a database server and configure the database / grants on it as part of the same Terraform run. Providers get initialized based on Terraform finding a resource owned by that provider in your Terraform code, and since this connection attempt happens when the provider gets initialized, you can't work around this with -target=<SPECIFIC RESOURCE>.
The workarounds I can think of would be to have a codebase for setting up the database server and a different codebase for setting up the database grants and suchlike ... or to have Terraform kick off a script that does that work for you (with dynamic parameters, of course!). Either way, you're effectively removing mysql_* resources from your initial Terraform run and that's what fixes this.
There are a couple of code changes that probably need to happen here - the Terraform MySQL provider would need to delay connecting to the database until Terraform tells it to run an operation on a resource, and it may be necessary to look at how Terraform handles dependencies across providers. I tried hacking in deferred connection logic just for the mysql_database resource to see if that solved all my problems and Terraform still complained about a dependency loop in the graph.
You can track the MySQL provider issue here:
https://github.com/terraform-providers/terraform-provider-mysql/issues/2
And the comments from before providers were split into their own releasable codebases:
https://github.com/hashicorp/terraform/issues/5687