Currently I have a aws infrastructure that is created manually. I want to import configuration of vpc and subnets. With the
terraform import aws_vpc.example vpc-id
I can get the cidr block of the vpc. Further with the cidr block I also want to get all the subnet ids as well. Is there a way to get all subnet ids with import command or I have to manually enter every id?
I cannot find any document where all subnet values can be imported? If there is please share
Thank you in advance!
If you already have vpc_id, then you can use aws_subnet_ids data to automatically get the ids of its subsets.
Example from docs:
data "aws_subnet_ids" "example" {
vpc_id = aws_vpc.example.vpc_id
}
Related
I am trying to obtain (via terraform) the dns name of a dynamically created VPCE endpoint using a data resource but the problem I am facing is the service name is not known until resources have been created. See notes below.
Is there any way of retrieving this information as a hard-coded service name just doesn’t work for automation?
e.g. this will not work as the service_name is dynamic
resource "aws_transfer_server" "sftp_lambda" {
count = local.vpc_lambda_enabled
domain = "S3"
identity_provider_type = "AWS_LAMBDA"
endpoint_type = "VPC"
protocols = ["SFTP"]
logging_role = var.loggingrole
function = var.lambda_idp_arn[count.index]
endpoint_details = {
security_group_ids = var.securitygroupids
subnet_ids = var.subnet_ids
vpc_id = var.vpc_id
}
tags = {
NAME = "tf-test-transfer-server"
ENV = "test"
}
}
data "aws_vpc_endpoint" "vpce" {
count = local.vpc_lambda_enabled
vpc_id = var.vpc_id
service_name = "com.amazonaws.transfer.server.c-001"
depends_on = [aws_transfer_server.sftp_lambda]
}
output "transfer_server_dnsentry" {
value = data.aws_vpc_endpoint.vpce.0.dns_entry[0].dns_name
}
Note: The VPCE was created automatically from an AWS SFTP transfer server resource that was configured with endpoint type of VPC (not VPC_ENDPOINT which is now deprecated). I had no control over the naming of the endpoint service name. It was all created in the background.
Minimum AWS provider version: 3.69.0 required.
Here is an example cloudformation script to setup an SFTP transfer server using Lambda as the IDP.
This will create the VPCE automatically.
So my aim here is to output the DNS name from the auto-created VPC endpoint using terraform if at all possible.
example setup in cloudFormation
data source: aws_vpc_endpoint
resource: aws_transfer_server
I had a response from Hashicorp Terraform Support on this and this is what they suggested:
you can get the service SFTP-Server-created-VPC-Endpoint by calling the following exported attribute of the vpc_endpoint_service resource [a].
NOTE: There are certain setups that causes AWS to create additional resources outside of what you configured. The AWS SFTP transfer service is one of them. This behavior is outside Terraform's control and more due to how AWS designed the service.
You can bring that VPC Endpoint back under Terraform's control however, by importing the VPC endpoint it creates on your behalf AFTER the transfer service has been created - via the VPCe ID [b].
If you want more ideas of pulling the service name from your current AWS setup, feel free to check out this example [c].
Hope that helps! Thank you.
[a] https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpc_endpoint_service#service_name
[b] https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpc_endpoint#import
[c] https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpc_endpoint#gateway-load-balancer-endpoint-type
There is a way forward like I shared earlier with the imports but it not going to be fully automated unfortunately.
Optionally, you can use a provisioner [1] and the aws ec2 describe-vpc-endpoint-services --service-names command [2] to get the service names you need.
I'm afraid that's the last workaround I can provide, as explained in our doc here [3] - which will explain how - as much as we'd like to, Terraform isn't able to solve all use-cases.
[1] https://www.terraform.io/language/resources/provisioners/remote-exec
[2] https://awscli.amazonaws.com/v2/documentation/api/latest/reference/ec2/describe-vpc-endpoint-services.html
[3] https://www.terraform.io/language/resources/provisioners/syntax
I've finally found the solution:
data "aws_vpc_endpoint" "transfer_server_vpce" {
count = local.is_enabled
vpc_id = var.vpc_id
filter {
name = "vpc-endpoint-id"
values = ["${aws_transfer_server.transfer_server[0].endpoint_details[0].vpc_endpoint_id}"]
}
}
I am trying to create a rule in the existing Default Route Table in OCI using terraform.
Basically I am trying to add a rule for internet gateway so I can access it using ssh. not sure but I am not able to access TCP till I am not adding rule in default table, new table not working for me..
But In OCI provider the option is available for only create new route table with rule instead of add rule in existing / default route table
I am just able to find below option for route table in oci provider, the rest belongs to DRG.
https://registry.terraform.io/providers/hashicorp/oci/latest/docs/resources/core_route_table
I am currently using below terraform code:
resource "oci_core_internet_gateway" "test_internet_gateway" {
#Required
compartment_id = var.compartment_ocid
vcn_id = oci_core_vcn.test_vcn.id
}
resource "oci_core_route_table" "test_route_table" {
#Required
compartment_id = var.compartment_ocid
vcn_id = oci_core_vcn.test_vcn.id
#display_name = "Default Route Table for xyz"
route_rules {
#Required
network_entity_id = oci_core_internet_gateway.test_internet_gateway.id
#cidr_block = "0.0.0.0/0"
destination = "0.0.0.0/0"
}
}
Any way around or solution will helps !!!!
Extended the answer of #bmuthuv
The below page have some clue that how we can Manage Default VCN Resources :
https://docs.oracle.com/en-us/iaas/Content/API/SDKDocs/terraformbestpractices_topic-vcndefaults.htm
I have replace resource oci_core_route_table with oci_core_default_route_table. The strange thing is resource "oci_core_default_route_table" is not present in registry provider document directly, you need to search "Managing Default VCN Resources" on oci register page as below:
https://registry.terraform.io/providers/oracle/oci/latest/docs
resource "oci_core_default_route_table" "this" {
#SOURCE PAGE : https://www.tfwriter.com/oci/r/oci_core_default_route_table.html
manage_default_resource_id = oci_core_subnet.test_subnet.route_table_id
route_rules {
#Required
network_entity_id = oci_core_internet_gateway.test_internet_gateway.id
destination = "0.0.0.0/0"
}
}
The displayed terraform code creates a Route Table and adds a route rule for 0.0.0.0/0. The missing piece is to map this Route Table to the subnet that is housing your VM.
Here are a couple of thoughts:
You create the entire VCN and Compute VM thereby you manage your infrastructure completely. This also enables to create a subnet along side the VCN and map the route table to it.
Use Terraform Resource discovery to generate TF code for existing infrastructure. Once the configuration files are generated, modify it to Map the Route Table to the subnet.
Lastly, please check this page to know about how to modify Default Resources. This could be your quick win.
UPDATE for OCI Provider
To answer your question, the version of OCI provider you are using (4.102.1) may not have the necessary functionality to create a route rules in a default route table in OCI. This could be due to a bug or limitation in the Terraform provider for OCI.
You can check the Terraform provider's documentation and GitHub issues to see if there are any known issues or workarounds for this problem. It's also possible that a newer version of the Terraform provider may have fixed this issue, so you may want to consider upgrading to the latest version if possible.
Alternatively, you may be able to add route rules in a default route table in OCI using other tools or methods, such as the OCI CLI or the OCI console.
I am trying to set up an EC2 with elastic IP with terraform. I am trying to use the existing VPC and subnets for the new EC2.
But Terraform is unable to recognise the existing subnet.
I am using the pre existing subnet like this -
variable "subnet_id" {}
data "aws_subnet" "my-subnet" {
id = "${var.subnet_id}"
}
When I run terraform plan I get this error -
Error: InvalidSubnetID.NotFound: The subnet ID 'subnet-02xxxxxxxxxx7' does not exist
status code: 400, request id: c4b6142b-5dfd-458c-959d-e5440b89c9fd
on ec2.tf line 3, in data "aws_subnet" "my-subnet":
3: data "aws_subnet" "my-subnet" {
This subnet was created by terraform in the past. So why does it say it doesn't exist?
Suggested debug:
Create 2 new terraform files:
First file, create a simple subnet (or VPC then subnet whatever)
Second file, try to retreive the subnet id like you posted.
The idea here is not to change anything else, meaning, same region, same creds, same everything.
Possible outputs:
1) you're not able to get the subnet ID - then you should be looking at things like the terraform version, provider version, stuff like that
2) you get the subnet ID, which means something in your creds, region, copy&paste of the name, basically human error is leading this blockade and you should revisit how you're doing things with emphysis on typos and permissions.
We can use the Data-Sources aws_vpc, aws_subnet and aws_subnet_ids:
data "aws_vpc" "default" {
default = true
}
data "aws_subnet_ids" "selected" {
vpc_id = data.aws_vpc.default.id
}
data "aws_subnet" "selected" {
for_each = data.aws_subnet_ids.selected.ids
id = each.value
}
And we can use them like in this LB example below:
resource "aws_alb" "alb" {
...
subnets = [for subnet in data.aws_subnet.selected : subnet.id]
...
}
This provides a reference to the VPC and subnets so you can pass the ID to other resources. Terraform does not manage the VPC or subnets when you do this, it simply references them.
It's irrelevant whether the subnet was initially created by Terraform or not.
The data source is attempting to find the subnet in the current state file. Your plan command returns the error because the subnet is not in your state file.
Try importing the subnet and then re-running the plan.
$ terraform import aws_subnet.public_subnet subnet-02xxxxxxxxxx7
Is there a way to create default AWS VPC using terraform?
aws_vpc resource doesn't have any attribute to mark VPC as default.
I know I can use aws-cli aws ec2 create-default-vpc, but that's outside of the terraform.
No it looks like you can't via TF, you can adopt an existing default VPC but not create it:
https://www.terraform.io/docs/providers/aws/r/default_vpc.html.
So use the AWS cli to recreate it (if you can) and then use this new provider to adopt it.
A default VPC comes with a public subnet in each Availability Zone, an internet gateway, and settings to enable DNS resolution.
As 17-August-2022, it is possible to create a default VPC through Terraform on AWS in the current region.
It is as simple as:
resource "aws_default_vpc" "default" {
tags = {
Name = "Default VPC"
}
}
Some important points to note: (As mentioned here)
The aws_default_vpc resource behaves differently from normal resources in that if a default VPC exists, Terraform does not create this resource, but instead "adopts" it into management. If no default VPC exists, Terraform creates a new default VPC, which leads to the implicit creation of other resources (e.g. Subnets, Internet Gateway etc.).
By default, terraform destroy does not delete the default VPC but does remove the resource from Terraform state. Set the force_destroy argument to true to delete the default VPC.
Is it possible to add a new VM only using Terraform? All the examples/samples and everything I have used Terraform for so far has me adding the VNet, Subnet, Network Interface, VM, Storage, etc., etc. all at the same time, referencing the resources crated within the script when creating other resources. For example, Terraform the Network Interface and then reference that when creating the VM.
What about if you already have the VNet, Subnet, etc. and just want to add a new, for example, Network Interface. Every time I try to do this and just reference what I think is the correct id, the plan stage works but then the apply fails with an "autorest:DoErrorUnlessStatusCode 400" error on he PUT call.
Is it just not possible to do this unless the resources were originally created using Terraform?
Yes you can. You can get the id from an created subnet with output. Like:
output "subnetid" {
value = "${azurerm_subnet.xxx.id}"
}
In your next template you can use this id in the subnet_id field.
The value of "${azurerm_subnet.xxx.id}" is base on the resourcegroup/vnet/subnet. So if you know how it is build you can also like to resource that are not created in terraform.
You can use the data directive to get existing resources as below. You can then interpolate these to create your VM.
data "azurerm_resource_group" "existing_deploy_rg" {
name = "RG"
}
data "azurerm_virtual_network" "existing_vnet" {
name = "existing-vnet"
resource_group_name = "${data.azurerm_resource_group.existing_deploy_rg.name}"
}
data "azurerm_subnet" "existing_subnet" {
name = "existing-subnet"
resource_group_name = "${data.azurerm_resource_group.existing_deploy_rg.name}"
virtual_network_name = "${data.azurerm_virtual_network.existing_vnet.name}"
}