We have reserved static (whitelisted) IP addresses that need to be assigned to a CloudNAT on GCP by terraform. The IPs are reserved and registered with a service provider, which takes weeks to get approved and added to their firewalls, so dynamic allocation is not an option.
The main problem for us is that the google_compute_router_nat section requires the nat_ip_allocate_option, but in this case the IP address has already been allocated, so it fails with an error stating exactly that. The only options for allocate are AUTO_ONLY and MANUAL_ONLY, but it seems maybe an EXISTING or RESERVED might be needed, unless I'm missing something obvious.
Here is the failing configuration:
resource "google_compute_address" "static_ip" {
name = "whitelisted-static-ip"
region = "${var.project_region}"
}
resource "google_compute_router_nat" "cluster-nat" {
name = "cluster-stg-nat"
router = "${google_compute_router.router.name}"
region = "${google_compute_router.router.region}"
nat_ip_allocate_option = "MANUAL_ONLY"
nat_ips = ["${google_compute_address.static_ip.self_link}"]
source_subnetwork_ip_ranges_to_nat = "LIST_OF_SUBNETWORKS"
subnetwork {
name = "${google_compute_subnetwork.service.self_link}"
source_ip_ranges_to_nat = ["ALL_IP_RANGES"]
}
}
Results in the following error:
Error: Error creating Address: googleapi: Error 409: The resource 'projects/staging-cluster/regions/us-central1/addresses/whitelisted-static-ip' already exists, alreadyExists
because the static IP resource is already reserved in GCP External IP Addresses and registered with the service provider.
Changing the google_compute_address resource to a data object was the magic. I modified it to be:
data "google_compute_address" "static_ip" {
name = "whitelisted-static-ip"
region = "${var.project_region}"
}
Where the name of "whitelisted-static-ip" is what we assigned to the reserved external IP address when we created it. The updated router NAT resource then became:
resource "google_compute_router_nat" "cluster-nat" {
name = "${var.cluster_name}-nat"
router = "${google_compute_router.router.name}"
region = "${google_compute_router.router.region}"
nat_ip_allocate_option = "MANUAL_ONLY"
nat_ips = ["${data.google_compute_address.static_ip.self_link}"]
source_subnetwork_ip_ranges_to_nat = "LIST_OF_SUBNETWORKS"
subnetwork {
name = "${google_compute_subnetwork.service.self_link}"
source_ip_ranges_to_nat = ["PRIMARY_IP_RANGE"]
}
}
which is only a mod to the nat_ips field to point to the data object. Simple two word change and we're good to go. Excellent!
It looks like the problem is with the google_compute_address resource, not the NAT. You are trying to create a resource that already exists. Instead you should do one of the following:
If you want Terraform to manage this resource for you import the resource into Terraform, see here https://www.terraform.io/docs/import/ and here https://www.terraform.io/docs/providers/google/r/compute_address.html#import
If you do not want Terraform to manage the IP address for you then you can use a data object instead of a resource object. This is essentially a read only resource lookup so that you can reference it in Terraform but manage it somewhere else. See here https://www.terraform.io/docs/configuration/data-sources.html and here https://www.terraform.io/docs/providers/google/d/datasource_compute_address.html
Related
I'm creating an azurerm_windows_virtual_machine resource that has a count which changes overtime.
resource "azurerm_windows_virtual_machine" "this" {
# The count variable could change
count = var.virtual_machine_count[terraform.workspace]
name = "vmt${count.index}"
...
tags = local.tags
}
After creating the VM resource, I'm creating a local_file that I would need to use as an Ansible inventory to configure the VMs.
resource "local_file" "ansible_inventory" {
depends_on = [azurerm_windows_virtual_machine.this]
content = templatefile(var.template_file_name,
{
private-ips = azurerm_windows_virtual_machine.this.*.private_ip_address,
ansible-user = var.ansible_user,
ansible-password = var.ansible_password,
ansible-ssh-port = var.ansible_ssh_port,
ansible-connection = var.ansible_connection,
ansible-winrm-transport = var.ansible_winrm_transport,
ansible-winrm-server-cert-validation = var.ansible_winrm_server_cert_validation
}
)
filename = var.template_output_file_name
}
The current behavior of this code is whenever I add another VM to the VM count, the local file resource adds the IP addresses of all VMs, including existing ones to the template file.
What I was hoping to do was to add the IP address of just the newly added VM resource to the template file.
Not sure if this is possible with Terraform only or if I need an external script to keep track of the existing and new IP addresses. Thanks.
I'm encountering a strange issue with Terraform's NSX-T provider. I'm referencing my Tier-0 router as a datasource, but I cannot use it in my resources.
Tier-0 Router data source:
data "nsxt_logical_tier0_router" "tier0_router" {
id = "c8a3e87c-ec49-4586-ad2e-ada799e0fd34"
}
Logical router resource:
# Router downlink
resource "nsxt_logical_router_downlink_port" "downlink_port" {
description = "DP1 provisioned by Terraform"
display_name = "vendor_bastion"
logical_router_id = nsxt_logical_tier0_router.tier0_router.id
linked_logical_switch_port_id = nsxt_logical_port.logical_port1.id
ip_address = "10.30.3.252/24"
}
It errors on the "nsxt_logical_tier0_router.tier0_router.id" line saying:
Error: Reference to undeclared resource
on tf-nsxt.tf line 46, in resource "nsxt_logical_router_downlink_port" "downlink_port":
46: logical_router_id = nsxt_logical_tier0_router.tier0_router.id
A managed resource "nsxt_logical_tier0_router" "tier0_router" has not been
declared in the root module.
The strange part is that if I put that ID on line 46 it works fine. Anyone seen this before?
According to the provider reference (https://www.terraform.io/docs/providers/nsxt/r/logical_router_downlink_port.html):
logical_router_downlink_port: This resource provides a means to define a downlink port on a logical router to connect a logical tier1 router to a logical switch. The result of this is to provide a default gateway to virtual machines running on the logical switch.
If you want co connect tier0 with tier1 you should use nsxt_logical_router_link_port_on_tier0 on this page: https://www.terraform.io/docs/providers/nsxt/r/logical_router_link_port_on_tier0.html
I figured this out and forgot to come back and post. The solution is that when you're referencing data, you need to add "data." in front of reference. So the correct way to reference the router is below:
# Router downlink
resource "nsxt_logical_router_downlink_port" "downlink_port" {
description = "DP1 provisioned by Terraform"
display_name = "vendor_bastion"
logical_router_id = data.nsxt_logical_tier0_router.tier0_router.id
linked_logical_switch_port_id = nsxt_logical_port.logical_port1.id
ip_address = "10.30.3.252/24"
}
I am sure this is a simple problem that I don't know how to interpret it at the moment.
I use 3 droplets(called rs) and have a template file which configures each.
[..]
data "template_file" "rsdata" {
template = file("files/rsdata.tmpl")
count = var.count_rs_nodes
vars = {
docker_version = var.docker_version
private_ip_rs = digitalocean_droplet.rs[count.index].ipv4_address_private
private_ip_mysql = digitalocean_droplet.mysql.ipv4_address_private
}
}
resource "digitalocean_droplet" "rs" {
count = var.count_rs_nodes
image = var.image
name = "${var.prefix}-rs-${count.index}"
region = var.region
size = var.rs_size
private_networking = true
user_data = data.template_file.rsdata.rendered
ssh_keys = var.ssh_keys
depends_on = ["digitalocean_droplet.mysql"]
}
[..]
When I do a terraform apply I get:
Error: Cycle: digitalocean_droplet.rs, data.template_file.rsdata
Note this is terraform 0.12
What am I doing wrong and how can I overcome this please?
This error is returned because the data.template_file.rsdata resource refers to the digitalocean_droplet.rs resource and vice-versa. That creates an impossible situation for Terraform: there is no ordering Terraform could use to process these that would ensure that all of the necessary data is available at each step.
The key problem is that the private IPv4 address for a droplet is allocated as part of creating it, but the user_data is used as part of that creation and so it cannot include the IPv4 address that is yet to be allocated.
The most straightforward way to deal with this would be to not include the droplet's IP address as part of its user_data and instead arrange for whatever software is processing that user_data to fetch the IP address of the host directly from the network interface at runtime. The kernel running in that droplet will know what the IP address is, so you can retrieve it from there in principle.
If for some reason including the IP addresses in the user_data is unavoidable (this can occur, for example, if there are a set of virtual machines that all need to be aware of each other) then a more complex alternative is to separate the allocation of the IP addresses from the creation of the instances. DigitalOcean doesn't have a mechanism to create private network interfaces separately from the droplets they belong to, so in this case it would be necessary to use public IP addresses via digitalocean_floating_ip, which may not be appropriate for all situations:
resource "digitalocean_floating_ip" "rs" {
count = var.count_rs_nodes
region = var.region
}
resource "digitalocean_droplet" "rs" {
count = var.count_rs_nodes
image = var.image
name = "${var.prefix}-rs-${count.index}"
region = var.region
size = var.rs_size
private_networking = true
ssh_keys = var.ssh_keys
user_data = templatefile("${path.module}/files/rsdata.tmpl", {
docker_version = var.docker_version
private_ip_rs = digitalocean_floating_ip.rs[count.index].ip_address
private_ip_mysql = digitalocean_droplet.mysql.ipv4_address_private
})
}
resource "digitalocean_floating_ip_assignment" "rs" {
count = var.count_rs_nodes
ip_address = digitalocean_floating_ip.rs[count.index].ip_address
droplet_id = digitalocean_droplet.rs[count.index].id
}
Because the "floating IP assignment" is created as a separate step after the droplet is launched, there will be some delay before the floating IP is associated with the instance and so whatever software is relying on that IP address must be resilient to running before the association is created.
Note that I also switched from using data "template_file" to the templatefile function because the data source is offered only for backward-compatibility with Terraform 0.11 configurations; the built-in function is the recommended way to render external template files in Terraform 0.12, and avoids the need for a separate configuration block.
I've been trying to get terraform to create a new ae interface with no luck.
My tf files are very basic working with a factory reset PA3020 that only has the user, password, and IP preconfigured.
It's connecting correctly as I've been able to create/modify other values such as a management profile.
Has anyone successfuly been able to create an aggregate group in paloalso using terraform? If so how was that done?
provider "panos" {
hostname = "${var.pa-mgt-ip}"
username = "${var.pa-username}"
password = "${var.pa-password}"
}
resource "panos_ethernet_interface" "ae_int1" {
name = "ae1"
vsys = "vsys1"
mode = "layer3"
comment = "AE interface from TF"
}
resource "panos_ethernet_interface" "phy_int1" {
name = "ethernet1/3"
vsys = "vsys1"
mode = "aggregate-group"
aggregate_group = "${panos_ethernet_interface.ae_int1.name}"
comment = "AE1 physical interface from TF"
}
resource "panos_ethernet_interface" "phy_int2" {
name = "ethernet1/4"
vsys = "vsys1"
mode = "aggregate-group"
aggregate_group = "${panos_ethernet_interface.ae_int1.name}"
comment = "AE1 physical interface from TF"
}
The error is ae1 'ae1' is not a valid reference and the interface is not getting created. If I manually create the ae1 interface in the UI and set the group to ae1 in for the physical interfaces in the TF file they fail with the error aggregate-group is invalid.
Does panos not currently support creating AE interfaces? I couldn't find any issues in github related to creating interfaces.
I've created an elastic beanstalk environment through terraform. I'd like to add a route53 record pointing at the load balancer's dns but I can't figure out how to get the full url from the outputs of the EB environment.
The aws_elastic_beanstalk_environment.xxx.load_balancers property contains the name but not the FQDN.
Once you have created both beanstalk resources like so
resource "aws_elastic_beanstalk_application" "eb_app" {
name = "eb_app"
description = "some description"
}
resource "aws_elastic_beanstalk_environment" "eb_env" {
name = "eb_env"
application = "${aws_elastic_beanstalk_application.eb_app.name}"
solution_stack_name = "64bit Amazon Linux 2015.03 v2.0.3 running Go 1.4"
setting {
namespace = "aws:ec2:vpc"
name = "VPCId"
value = "something"
}
setting {
namespace = "aws:ec2:vpc"
name = "Subnets"
value = "somethingelse"
}
}
The documentation included here specifies that you can use 'cname' to output the fully qualified DNS name for the environment
To do that you would write something like
output "cname" {
value = "${aws_elastic_beanstalk_environment.eb_env.cname}"
}
Then, in the same directory that you invoked the module, you could
cname_to_pass_to_route53 = "${module.some_eb_module.cname}"
If the cname is not the exact verision of the url that you need, you could append or prepend the variable name when passing it in. It does say fully qualified DNS name though, so I don't think you'd need to do that.
cname_to_pass_to_route53 = "maybeHere://${module.some_eb_module.cname}/orOverHere"