I have a Terraform 0.11 project with 30-40 different resources. I would like to delete all of them except a few - and those few are logically related to each other.
I was looking for something close to terraform destroy --except=resource-id but that of course doesn't exist.
Is there a way to achieve that without too much scripting (Terraform admins have various OSs)? Would using modules make that process easier perhaps?
There is no --except feature in terraform destroy command currently. If you really want to do that, and you know what you are doing, here is the workaround.
# list all resources
terraform state list
# remove that resource you don't want to destroy
# you can add more to be excluded if required
terraform state rm <resource_to_be_deleted>
# destroy the whole stack except above excluded resource(s)
terraform destroy
So why do these commands work for your idea?
The state (*.tfstate) is used by Terraform to map real world resources to your configuration, keep track of metadata.
terraform state rm cleans a record (resource) from the state file (*.tfstate) only. It doesn't destroy the real resource.
Since you don't run terraform apply or terraform refresh, after terraform state rm, terraform doesn't know the excluded resource was created at all.
When you run terraform destroy, it has no detail about that excluded resource’s state and will not destroy it. It will destroy the rest.
By the way, later you still have chance to import the resource back with terraform import command if you want.
Targetting each resource (while skipping over the data resources) except the one you want is probably the only way atm:
#! /bin/bash
while read -r resource; do
terraform destroy -target="$resource"
done < <(terraform state list | grep -vE "^data\." | grep -vE "dont_remove|also_important")
I have a bit of a different work around. The resources I do not want to delete with "terraform destroy" I create as "null_resource" using a provisioner with CLI. You can still use your variables in terraform as well.
for example (Create a resource group, but it is persistent due to null_resource)
resource "null_resource" "backend-config" {
provisioner "local-exec" {
command = <<EOT
az group create --location ${var.Location} --name ${var.Resource_group_name} --tags 'LineOfBusiness=${var.Lob}' 'Region=${var.Region}' 'Purpose="Terraform-Primary-Resource-Group-${var.Lob}'
EOT
interpreter = ["Powershell", "-Command"]
}
}
Now if you destroy the resources using terraform destroy.
Any null_resource will remain intact.
To delete all your stack except some resources, you need to:
Create a backup of your current state : terraform state pull > bkup.json
List all your resources : terraform state list
"rm" the resources that you want to keep : terraform state rm "aws_myresource". It will not delete the resource in your cloud, but only from the terraform perspective.
Destroy your stack : it will be delete from the cloud all your resources except the ones you "rm" just before. terraform destroy
You now have an empty state (0 resources). Save it in order to be able to edit it : terraform state pull > state-to-edit.json
Edit the state-to-edit.json by adding resources object {} from the bkup.json in the .resources[] block. You also need to increment by 1 the .serial value
Push the modified state : terraform state push ./state-to-edit.json
You can verify with terraform plan or terraform state list that you still have the wanted resources & deleted all the others.
List resources:
terraform state list
data.terraform_remote_state.rg
azurerm_postgresql_database.postgresql_database
azurerm_postgresql_server.postgresql_server
Remove resource
terraform destroy -target azurerm_postgresql_database.postgresql_database -auto-approve
BMW's answer is best if you just need to destroy things, and not change the code. That is, if you intend to bring those resources back up at some later time.
If you just want to remove the resources, the proper solution is to remove the Terraform definitions you wish to destroy, and then do a normal terraform apply. Just as you would for adding a resource.
(An old question, I know, but I was surprised to not see anyone mention alternative. The OP's use case seems to be covered by the other answers, so this answer is for others stumbling in here.)
terraform destroy -target RESOURCE_TYPE.NAME -target RESOURCE_TYPE2.NAME
Related
I have recently started working on Terraform, have a question on terraform state mv and terraform import. As per the documentation, terraform state mv can be used when a resource name changes, and the updated name has to be added to the state file. And terraform import can be used to import the resources created outside of Terraform to a state file. My question is even when a resource name changes or code structure changes(using modules), we can still use terraform import to update the state file correct? Could anyone tell me, what is the real benefit of using terraform state mv command?
So the question really is this particular case:
I have renamed the TF resource / changed the structure of the resource
in IaC. Can I just re-import it into the new structure, instead of moving it?
Yes you can, but what will happen to the state? You'll be importing a resource you're already managing according to the TF state. The old resource that you've modified should still be managed, therefore you might run into issues where the TF operator will attempt to recreate it or even delete it. It will all depend on what state matches the reality in your cloud provider.
If you'd like to still import the updated, I'd go for terraform state rm & terraform import afterwards. This is sometimes required / an easy hack after big changes to a particular module / resource. It's also a good debugging experience, when you're not exactly sure about how does the cloud resource matches the TF code, as you're see state differences only for this newly imported resource.
One benefit of terraform state mv is useful if you need to refactor your code in or out of modules. I've used it quite a bit. I recommend backing up your state before making any changes. If you are using a remote state, you can always take a copy of it, disable your use of the remote state temporarily and then utilize the copy locally.
You can see the names of your state objects by using terraform state list.
The usage of terraform import is to add an existing thing to your state file, so it's tracked.
Terraform Import - Terraform is able to import existing infrastructure. This allows you take resources you've created by some other means and bring it under Terraform management.
Terraform State MV - It is less common situation where you wish to retain an existing remote object but track it as a different resource instance address in Terraform, such as if you have renamed a resource block or you have moved it into a different module in your configuration.
Use terraform import for all resources, created outside terraform
Use terraform state mv in the case, you want to restruct a already exisiting terraform resource.
I am using terraform state mv as soon as my projects needs to be restructed, e.g. become more complex, want to move to modules, etc.
Sometimes (even for older terraform projects), it could also be a good practice to import the resource again (with another name) and to a terraform state rm.
I'm new to this terraform world and I've been assigned into the task of creating many configurations to azure with it.
I'm developing a main.tf script (which creates some resources, like resource group, vnets, kubernetes cluster, app services, etc.) and while coding it and executing
Terraform apply, it seems to only apply what changed doing in fact updates.
Then we deleted the resource group the script created and a colegue of mine had to run the same script with terraform creating a resource group with another name since i didn't had a required permission, after that, if i run the command Terraform apply it fails and gives errors, that say that the resource cannot be created because it already exists.
After reading some documentation i found that it might be because of the state
https://www.terraform.io/docs/state/index.html
Is the update of a script something that only works for each session of terraform?
Even doing a Terraform refresh doesn't seem to work.
Or probably I'm just mistaking and there is no way to update some resources.
EDIT: for some reason the state file that was on the storage only had a few things, the solution was to delete everything and create again.
For the new resources, there is nothing more, the Terraform script helps you create the resources you set in the script.
For the existing resources, when you make changes in the script that you already deployed via the Terraform, then it will check the state file to make sure what changes the resources should update. If there is no state file ( or you delete it), then it will deploy the Terraform script directly, but if any resources you want to deploy already exists, then it will fail due to the existing resources. And the command terraform refresh just updates the last state of the resources in the Terraform script that you already deployed. If the deployment failed and the state file has no resources in it, then refresh is not useful.
If someone else ran terraform apply for you because you didn't have access, and now you want to modify that terraform and run it yourself, you need to get the state file that was generated when that other person ran it. You absolutely have to maintain the Terraform state file somewhere, so that it can be accessed on subsequent runs. You should really configure a Terraform backend, instead of using local state files.
You need to be aware that Terraform stores everything it does in the state file, and refers to that file before every run. A terraform refresh only tells Terraform to refresh the state of the things that are in the state file, it doesn't rebuild the state file from scratch. Understanding Terraform state files is so fundamental to the use of Terraform that you really need to understand this before using it.
I am new to terraform and have been playing with it for a while. In the scenario I am trying to execute, I am trying to create a stack for each deployment of mine. I am able to create the resources I need. However, when I want to deploy a new stack, it is trying to destroy the already existing stack. I have tried to use the lifecycle key with prevent destroy, but that throws an error saying the resource cannot be deleted.
the plan would destroy this resource, but it currently has lifecycle.prevent_destroy set to true. To avoid this error and continue with the plan, either disable lifecycle.prevent_destroy or adjust the scope of the plan using the -target flag.
Is there no way to skip or make terraform "forget" the resource it created? I understand this is tricky because of the state file that terraform maintains, but any leads would help.
Thanks in advance.
You can remove resources from the state using terraform state rm <resource>, maybe that's what you are looking for?
More information available at https://www.terraform.io/docs/commands/state/rm.html.
I think you need to use terraform workspaces for different deployments (environments ?). So you will be able to create different resources for different envs (this is mentioned in this comment Terraform forget resource and recreate?)
Information: https://www.terraform.io/docs/state/workspaces.html
It takes a long time to run terraform and wait.
So I would like to run it to exclude rds that takes the longest time to excute
or I would like to run only ec2 resource.
Is there a way to do such things in terraform?
You can use -target=resource like this:
terraform plan -target=module.mymodule.aws_instance.myinstance
terraform apply -target=module.mymodule.aws_instance.myinstance
or
terraform plan -target=aws_instance.myinstance
terraform apply -target=aws_instance.myinstance
Disclaimer: Before downvoting the answer, please note that he actually asked to either "exclude" or "run only ec2 resource". And after all this time the exclude feature request is still open in the terraform repo.
Adding to Julio's answer, you could target multiple resources in the following manner:
terraform init
terraform plan -target=resource_type1.resource_name1 -target=resource_type2.resource_name1
terraform apply -target=resource_type1.resource_name1 -target=resource_type2.resource_name1
I would like to run it to exclude rds that takes the longest time
Terraform currently does not support exclusion of resources (aka. inverse targeting).
Issue #2253: "feature request: inverse targeting / exclude"
(Thanks for the link Julio Daniel Reyes.)
For those who use variables
terraform plan -var-file dev.tfvars -target=module.mymodule.aws_instance.myinstance
terraform apply -var-file dev.tfvars -target=module.mymodule.aws_instance.myinstance
Is terraform destroy needed before terraform apply? If not, what is a workflow you follow when updating existing infrastructure and how do you decide if destroy is needed?
That would be pretty non-standard, in my opinion. Terraform destroy is only used in cases where you want to completely wipe your infrastructure. One of the biggest features of terraform is that it can do an intelligent delta of your desired infrastructure and your existing infrastructure and only make the changes needed. By performing a refresh, plan and apply you can ensure that terraform:
refresh - Has an up-to-date understanding of your current infrastructure. This is important in case anything was changed manually, outside of your terraform script.
plan - Prepares a list for you to review of what terraform intends to modify, or delete (or leave alone).
apply - Performs the changes laid out in the plan.
By executing these 3 commands in sequence terraform will only perform the changes necessary, in the order required, to bring your environments in line with any changes to your terraform file.
Where I find destroy to be useful is in non-production environments or in cases where you are performing a restructure that's so invasive that starting from scratch would ensure a safer build.
*There are also edge cases where terraform may fail to understand the correct order of operations (do I modify a security group first or a security group rule?), or it will find itself in a dependency cycle and will be unable to perform an operation. In those cases, however, running destroy is a nuclear solution. In general, I would perform the problem change manually (via command line, or AWS Console, if I'm in AWS), to nudge it along and then run a refresh, plan, apply sequence to get back on track.
No terraform destroy is not needed before terraform apply.
Your Terraform configuration (*.tf and *.tfvars files) describes the desired state of your infrastructure. It says "this is how I want my infrastructure to be."
You use the terraform tool to plan and apply changes to get your infrastructure into the desired state you have described. You can make those changes incrementally without destroying anything.
A typical workflow might be:
make changes to .tf and .tfvars files
refresh state
plan changes
review the planned changes
apply those changes
If you wanted to completely destroy that infrastructure you'd use terraform plan -destroy to see what Terraform intends to destroy. If you are happy with that you'd then use terraform destroy to destroy it.
Typically, destroy is rarely used, unless you are provisioning infrastructure for a temporary purpose (e.g., builds) or testing your ability to provision from a clean slate with different parameters. Even then, you could use a count parameter on resources to temporarily provision resources by increasing the count, then decreasing it again when no longer needed.
More comments after #mwielbut's answer.
Instead of option apply + destroy, you need to run terraform with option taint + apply
Normally we don't need run terraform destroy at all. It is a really dangerous option, especially for a production environment.
with option plan and apply, it is good enough to update the infrastructure with code.
But if you do need to destroy some resources and re-build something which is already created, you can use the option of taint, which is the right answer for your question, it is so important and missed in #mwielbut's answer.
The terraform taint command manually marks a Terraform-managed resource as tainted, forcing it to be destroyed and recreated on the next apply.
This command will not modify infrastructure but does modify the state file in order to mark a resource as tainted. Once a resource is marked as tainted, the next plan will show that the resource will be destroyed and recreated and the next apply will implement this change.
Refer:
command taint:
https://www.terraform.io/docs/commands/taint.html
a sample of option taint:
https://www.terraform.io/docs/modules/usage.html
Terraform destroy destroys all the resources and it is not required if you want to apply incremental changes. Destroy should be only used if you want to destroy the whole infrastructure.
No need to use the destroy command before apply. as long as you are in testing period you can use destroy command or destroy the complete infra you can use destroy command
You can use below flow
terraform init
terraform plan
terraform apply
if you made any manual changes that needs to update in your state file, use below command to update the state file.
Terrafrom refresh
You don't need to run to terraform destroy . If you have made any changes to you infrastructure, [added/ removed a resource], on next terraform plan & terraform apply, the changes will be reflected automatically
Terraform apply always refreshes the Terraform state, so if you change anything, it auto recognizes the changes, lets say you've updated your NSG rules, added new VM, deleted old VM, so when you run terraform apply again, your old state gets updated with the new state where you've Added/Updated/Deleted.
If you use terraform destroy, it just kills the entire state and you'll be back to the new state if you are running terraform apply.
You need to use terraform destroy only if you think you just want to bring down your infrastructure and you don't really need it.
For minor - major changes like Adding Components, Updating Rules, Deleting other things, you can use plan and apply without any problem.
Simply NO.
You don't need to run terraform apply before terraform destroyو Your terraform (.tf) files describe the state of your infrastructure.
terraform apply always refresh your infrastructure. And it identifies the state of infrastructure and updates it.
terraform destroy only use is to bring down and completely wipe down your infrastructure. (You have to think twice before using it) you can use terraform plan and terraform refresh to ensure the state of the infrastructure.
You could always manually destroy your instances, after only running your terraform apply. Then when you run terraform apply it will create brand new instances without the terraform destroy.
No! you don't need to run terraform destroy when you need a modification of resources! This is the beauty of Infra-as-Code.
Here are some more details on Terraform init, plan, apply and destroy -
terraform init command is used to initialize a working directory containing Terraform configuration files. This is the first command that should be run after writing a new Terraform configuration or cloning an existing one from version control. It is safe to run this command multiple times.
terraform plan command creates an execution plan. By default, creating a plan consists of:
a) Reading the current state of any already-existing remote objects to make sure that the Terraform state is up-to-date.
b) Comparing the current configuration to the prior state and noting any differences.
c) Proposing a set of change actions that should, if applied, make the remote objects match the configuration.
terraform apply command executes the actions proposed in a Terraform plan. (you can do an apply without plan however it's not a best practice)
terraform destroy command is a convenient way to destroy all remote objects managed by a particular Terraform configuration.
Core Terraform workflows:
The core Terraform workflow has five steps:
Write - Author infrastructure as code.
Terraform init - it’ll automatically download and install partner and community provider directly to the local disk so that it can be used by other commands Plugin_Installation, Backend_Initialization, ChildModule_Installation and Community and third party plugin
Terraform plan - Preview changes before applying.
Terraform Apply - Provision reproducible infrastructure.
Terraform destroy - It will destroy your infrastructure.
No need for terraform destroy, as it will just destroy all the resources created.
You just need to provide the backend configuration in your tf file.
Backend configuration is the configuration in order to retrieve terraform state files.
Terraform apply first time will create your cloud infrastructure, this will update your state file also.
And next apply terraform will compare what new/update resources are to be done with what is already there using state file and will deploy accordingly.