I want to divide my infrastructure into two parts:
Persistent (firewalls, block storages, etc)
Dynamic (that will consume persistent resources from #1)
I want to be sure that persistent part never would be deleted and at the same time, there would be an option terraform destroy on the dynamic infrastructure part.
All resources you do not want to destroy you have to add the lifecycle policy: prevent_destroy
Have a look at the documenation: https://www.terraform.io/docs/configuration/resources.html#prevent_destroy
To fully prevent the destruction you would have to fine tune the permissions on the resources at your provider. However there is an easy way to divide your infrastructure.
Terraform offers a remote state data source which allows you to use output from a different project so you won't be able to destroy those resources while working with the dynamic part.
I have a bit of a different work around. The resources I do not want to delete with "terraform destroy" I create as "null_resource" using a provisioner with CLI. You can still use your variables in terraform as well.
for example (Create a resource group, but it is persistent due to null_resource)
resource "null_resource" "backend-config" {
provisioner "local-exec" {
command = <<EOT
az group create --location ${var.Location} --name ${var.Resource_group_name} --tags 'LineOfBusiness=${var.Lob}' 'Region=${var.Region}' 'Purpose="Terraform-Primary-Resource-Group-${var.Lob}'
EOT
interpreter = ["Powershell", "-Command"]
}
}
Now if you destroy the resources using terraform destroy. Any null_resource will remain intact.
I would solve this by having 2 Terraform deployments. You create the "static" resources once, and don't touch them. For extra safety, manually add a deletion lock to those resources (eg. I know you can do this in Azure, I assume other cloud providers have a similar solution).
Import these resources in your Dynamic Terraform deployment, using data blocks (not resources). Terraform will never attempt to delete resources you import using data blocks.
Related
I am currently writing a Terraform file which should create all required resources in Azure in a specific resource group. As I do not have the permissions to create resource groups I imported an existing resource group following this link.
Now the problem is that when I run terraform destroy, the system also wants to delete the resource group as it was included as a resource.
Is there a simple way I can tell Terraform to destroy all created resources except the imported resource group? Or another way to approach this problem?
I tried the lifecycle variable prevent_destroy but it prevents any resource from being destroyed.
The solution presented here does not really work for me as I do not want to use multiple commands as this adds the risk of human error. What I am basically looking for is a way to label the resource group in the Terraform file in a way that it is not deleted.
If you want to refer to a resource but not have Terraform manage the lifecycle of it (either at all or just in that specific Terraform workspace) then you can use data sources.
In your case you could replace the azure_resource_group resource with the azure_resource_group data source.
I have terraform managing my infrastructure in Azure. However, there are cases where the state can get out of sync when other services are changing the infrastructure as well.
For example, I have terraform create an Application Gateway. But I also have an AKS cluster with AGIC enabled, which dynamically updates/changes rules, listeners, etc. inside of Application Gateway. So if terraform is re-run after AGIC makes some changes, terraform doesn't know and wants to reset to the default config it knows about.
Maybe this isn't possible, but is there an automated way to sync the two? It's kind of unfeasible to have to go into the terraform config and manually add the changes AGIC makes every time it does so. At this point, is it even worth managing Application Gateway with terraform?
If you really want to create the Application Gateway with Terraform, you should probably use the Lifecycle Meta-Argument ignore_changes on the attributes that will be modified by AKS. It is not perfect, but at least they can share their responsibilities without overwriting each other.
The ignore_changes feature is intended to be used when a resource is
created with references to data that may change in the future, but
should not affect said resource after its creation. In some rare
cases, settings of a remote object are modified by processes outside
of Terraform, which Terraform would then attempt to "fix" on the next
run. In order to make Terraform share management responsibilities of a
single object with a separate process, the ignore_changes
meta-argument specifies resource attributes that Terraform should
ignore when planning updates to the associated remote object.
For example :
resource "azurerm_application_gateway" "example" {
...
lifecycle {
ignore_changes = [
http_listener,
request_routing_rule,
backend_http_settings
]
}
...
}
I know that when I do terraform apply it does not deploy a resource if the previous deployment within the same terraform state, it would not re-create it .
But I want to do something different:
Create a resource if it is not created by someone else.
But if the resource is already there and even it is not in the terraform state, do not generate an error and have refrence to its name.
Is there any known pattern to do this?
By design Terraform providers will typically not automatically "adopt" existing objects as now being managed by Terraform, because to do so would potentially lead to costly mistakes if you inadvertently bind a remote object to a Terraform resource and then run terraform destroy without realizing what is going to be destroyed.
Instead, you must bind existing objects to your Terraform resources using the terraform import command, telling Terraform explicitly that you intend it to become the sole manager of that object.
This question is not how to import and it's not what's the purpose of tfstate. It's what's the purpose of importing a pre-existing resource, esp. compared to just referencing the ID of the existing resource?
Terraform has the feature of terraform import. HashiCorp describes the purpose of this as:
Terraform is able to import existing infrastructure. This allows you take resources you've created by some other means and bring it under Terraform management.
This is a great way to slowly transition infrastructure to Terraform, or to be able to be confident that you can use Terraform in the future if it potentially doesn't support every feature you need today.
I read the article about the purpose of Terraform state. It does make sense to me to track Terraform state with .tfstate files when those files are mappings back to the configurations in .tf files.
But it's still unclear to me what the purpose of a standalone .tfstate file is when it only maps to an empty resource block. If there is a resource not in terraform yet, I would typically do one of two things:
put the resource in terraform, tear down the resource manually and re-deploy the resource with terraform, or...
keep the resource un-templated, reference its resource ID as a parameter and get its metadata via a data element for terraform-managed resources that rely on it.
Is terraform import an alternative to those two approaches? And if so, why would you use that approach?
The only way to make changes to an imported resource (that only has an empty resource block in the .tf file and detailed state in .tfstate) is to make manual changes and then re-import into .tfstate`, right? And if so, then what's the point of tracking the state of that resource in terraform?
I'm sure there's a good reasons. Just want to understand this deeper! Thanks!
But it's still unclear to me what the purpose of a standalone .tfstate
file is when it only maps to an empty resource block.
You wouldn't use a standalone .tfstate file. You would be using the same .tfstate file that all your other resources are in.
If there is a resource not in terraform yet, I would typically do one
of two things:
put the resource in terraform, tear down the resource manually and re-deploy the resource with terraform, or...
keep the resource un-templated, reference its resource ID as a parameter and get its metadata via a data element for
terraform-managed resources that rely on it.
Is terraform import an alternative to those two approaches? And if so,
why would you use that approach?
Consider the case where you have a production database with terrabytes of data already load in it, and users actively performing actions that query that database 24 hours a day. Your option 1 would require some down time, possibly a lot of down time, because you would have to deal with backing up and restoring terrabytes of data. Your option 2 would never let you manage changes to your database server via Terraform. That's what the Terraform import feature solves. It lets Terraform take "full control" of resources that already exist, without having to recreate them.
I agree that if a system outage is not an issue, and if recreating a resource isn't going to take much time, using option 1 is the way to go. Option 2 is only for resources that you never want to fully manage in Terraform, which is really a separate issue from the one Terraform import solves.
When importing a resource with terraform import it is necessary to write the configuration block to manage it with Terraform. On the same page you linked it states:
The current implementation of Terraform import can only import resources into the state. It does not generate configuration. A future version of Terraform will also generate configuration.
Because of this, prior to running terraform import it is necessary to
write manually a resource configuration block for the resource, to
which the imported object will be mapped.
So to bring preexisting resources under Terraform management, you first write the resource block for it in a .tf file. Next you use terraform import to map the resource to this resource block in your .tfstate. The next time you run terraform plan, Terraform will determine what changes (if any) will need to be made upon the next terraform apply based on the resource block and the actual state of the resource.
EDIT
The "why" of terraform import is to manage resources that are previously unknown to Terraform. As you alluded to in your second bullet point, if you want metadata from a resource but do not want to change the configuration of the resource, you would use a data block and reference that in dependent resources.
When you want to manage the configuration of a resource that was provisioned outside of Terraform you use terraform import. If you tear down the resource there may be data loss or service downtime until you re-deploy with Terraform, but if you use terraform import the resource will be preserved.
The import process can be started with an empty resource block, but the attributes need to be filled out to describe the resource. You will get the benefits of terraform plan after importing, which can help you find the discrepancies between the resource block and the actual state of the resource. Once the two match up, you can continue to make additional changes to the resource like any other resource in Terraform.
Terraform state file is your source of truth for your cloud infrastructure. Terraform uses local state to create plans and make changes to the infrastructure. Before any terraform operation, terraform does a refresh to update the state with the real infrastructure.
Can we destroy a particular resource.
For example : An azure sql database only, without affecting the sql server or any firewalls.
will the below work and what is the resource address.
terraform destroy -target xxx
yes terraform has that functionality to destroy selected resources, but first you have to detached the dependent resources from the target resource and then try this command terraform destroy -target RESOURCE_TYPE.NAME
Yes, you can destroy specific resources, one at a time.
Following the terraform azure sql example : https://www.terraform.io/docs/providers/azurerm/r/sql_database.html
When the resources are created, they are registered in the terraform state file.
You can list the resources in the state file :
$ terraform state list
azurerm_resource_group.test
azurerm_sql_database.test
azurerm_sql_server.test
You can then destroy the sql database only with this command :
$ terraform destroy -target=azurerm_sql_database.test