Does terraform support aws backup feature for to restore the image from vault (https://www.terraform.io/docs/providers/aws/r/backup_plan.html )?
As I read the document I can see that it does support creating of backup plan, assigning resources and policy, creating vault but doesnot support restore of an image or ebs volume
How do i add the restore block in my terraform template
Terraform's execution model is designed for translating declarative descriptions of an intended state into imperative actions to reach that state automatically, and so its model doesn't really support "exceptional" processes like restoring backups.
However, you can develop a process for restoring backups alongside Terraform whereby the main restore action is done using the AWS Console, AWS CLI, or API in your own automation, and then you inform Terraform after the fact that it should use the restored object via its state manipulation commands.
For example, if you have an EBS volume managed by Terraform using an aws_ebs_volume resource, you might also use Terraform to configure an AWS Backup plan for that volume, and then backups will be created automatically as per your plan.
In the exceptional situation where your existing volume is lost or corrupted and you want to restore the backup, the person responding to the incident can follow the following process:
Create an AWS Backup restore job either using the AWS Console, the AWS CLI, or some software of your own design using the AWS Backup API.
Once the backup job is complete, consult the CreatedResourceARN to find the id if the new object that was created by restoring the backup. In the case of an EBS volume, this will be the final part of the after the :volume/ separator.
Tell Terraform to "forget" the existing EBS volume object that is now destroyed or damaged:
terraform state rm aws_ebs_volume.example
Tell Terraform to import the object created by restoring the backup as the new remote object associated with the Terraform resource:
terraform import aws_ebs_volume.example vol-049df61146c4d7901
If your old EBS volume is still present but corrupted or otherwise damaged, the final step would be to locate and manually destroy the remant of it, because Terraform is no longer managing it and therefore it would otherwise be left in place forever.
After this process is complete, Terraform will consider the new object to be the one managed by that resource, and you can use Terraform as normal with that resource moving forward. The same principle applies to any of the object types supported by AWS Backup, as long as they have a resource type in the AWS provider that supports terraform import.
Related
I am using Terraform scripts to create azure services, I am having some doubts regarding Terraform,
1) If I have one environment let say dev in azure having some azure resources how can I copy all the resources to new environment lest say prod using terraform script.
2)what are the impact of re-run the terraform file with additional azure resources, what it will do.
3)What if I want to create an app service with the same name from Terraform script that already present in the azure will it update the resource or do nothing after terraform execution completed.
Please feel free to answer the question, it will be great help.
To answer your questions:
You could create a new workspace with terraform workspace new and copy all configuration files (.tf) to the new environment, then run terraform init, plan, apply.
The terraform will compare the content in your current state file with your configuration file, then update the new attributes or creating new resources other than re-creating the existing resources.
You could run terraform import to import existing infrastructure into Terraform. For referencing existing resources in the portal, you can use data sources.
This question is not how to import and it's not what's the purpose of tfstate. It's what's the purpose of importing a pre-existing resource, esp. compared to just referencing the ID of the existing resource?
Terraform has the feature of terraform import. HashiCorp describes the purpose of this as:
Terraform is able to import existing infrastructure. This allows you take resources you've created by some other means and bring it under Terraform management.
This is a great way to slowly transition infrastructure to Terraform, or to be able to be confident that you can use Terraform in the future if it potentially doesn't support every feature you need today.
I read the article about the purpose of Terraform state. It does make sense to me to track Terraform state with .tfstate files when those files are mappings back to the configurations in .tf files.
But it's still unclear to me what the purpose of a standalone .tfstate file is when it only maps to an empty resource block. If there is a resource not in terraform yet, I would typically do one of two things:
put the resource in terraform, tear down the resource manually and re-deploy the resource with terraform, or...
keep the resource un-templated, reference its resource ID as a parameter and get its metadata via a data element for terraform-managed resources that rely on it.
Is terraform import an alternative to those two approaches? And if so, why would you use that approach?
The only way to make changes to an imported resource (that only has an empty resource block in the .tf file and detailed state in .tfstate) is to make manual changes and then re-import into .tfstate`, right? And if so, then what's the point of tracking the state of that resource in terraform?
I'm sure there's a good reasons. Just want to understand this deeper! Thanks!
But it's still unclear to me what the purpose of a standalone .tfstate
file is when it only maps to an empty resource block.
You wouldn't use a standalone .tfstate file. You would be using the same .tfstate file that all your other resources are in.
If there is a resource not in terraform yet, I would typically do one
of two things:
put the resource in terraform, tear down the resource manually and re-deploy the resource with terraform, or...
keep the resource un-templated, reference its resource ID as a parameter and get its metadata via a data element for
terraform-managed resources that rely on it.
Is terraform import an alternative to those two approaches? And if so,
why would you use that approach?
Consider the case where you have a production database with terrabytes of data already load in it, and users actively performing actions that query that database 24 hours a day. Your option 1 would require some down time, possibly a lot of down time, because you would have to deal with backing up and restoring terrabytes of data. Your option 2 would never let you manage changes to your database server via Terraform. That's what the Terraform import feature solves. It lets Terraform take "full control" of resources that already exist, without having to recreate them.
I agree that if a system outage is not an issue, and if recreating a resource isn't going to take much time, using option 1 is the way to go. Option 2 is only for resources that you never want to fully manage in Terraform, which is really a separate issue from the one Terraform import solves.
When importing a resource with terraform import it is necessary to write the configuration block to manage it with Terraform. On the same page you linked it states:
The current implementation of Terraform import can only import resources into the state. It does not generate configuration. A future version of Terraform will also generate configuration.
Because of this, prior to running terraform import it is necessary to
write manually a resource configuration block for the resource, to
which the imported object will be mapped.
So to bring preexisting resources under Terraform management, you first write the resource block for it in a .tf file. Next you use terraform import to map the resource to this resource block in your .tfstate. The next time you run terraform plan, Terraform will determine what changes (if any) will need to be made upon the next terraform apply based on the resource block and the actual state of the resource.
EDIT
The "why" of terraform import is to manage resources that are previously unknown to Terraform. As you alluded to in your second bullet point, if you want metadata from a resource but do not want to change the configuration of the resource, you would use a data block and reference that in dependent resources.
When you want to manage the configuration of a resource that was provisioned outside of Terraform you use terraform import. If you tear down the resource there may be data loss or service downtime until you re-deploy with Terraform, but if you use terraform import the resource will be preserved.
The import process can be started with an empty resource block, but the attributes need to be filled out to describe the resource. You will get the benefits of terraform plan after importing, which can help you find the discrepancies between the resource block and the actual state of the resource. Once the two match up, you can continue to make additional changes to the resource like any other resource in Terraform.
Terraform state file is your source of truth for your cloud infrastructure. Terraform uses local state to create plans and make changes to the infrastructure. Before any terraform operation, terraform does a refresh to update the state with the real infrastructure.
I am using terraform to deploy to multiple AWS accounts and each account with its own set of environments. I'm using terraform workspaces and s3 remote state. When I switch between these accounts my terraform workspace list is empty now for one of the accounts. Is there a way to sync the state of workspace from the s3 remote state?
Please advise.
Thanks,
I have tried to create the workspace but when I run terraform plan it does create all the resources even though they exists already in the remote state.
I managed to fix it using the following:
I created the new namespaces manually using terraform workspace command
terraform workspace new dev
Created and switched to workspace "dev"!
You're now on a new, empty workspace. Workspaces isolate their state,
so if you run "terraform plan" Terraform will not see any existing state
for this configuration.
I went to S3 where I have the remote state and now under the environment dev I have duplicate states.
I copied the state from the old folder key and added to the new folder key (using copy/paste) in S3 console window
IN dynamo db lock state I have duplicate id of LockID for my environment with different digests. I had to copy the Digest of the old entry and replace the digest for the new entry. After that when I run terraform plan everything went smoothly and I had to repeat the same process for all the environments.
I hope this helps anyone else having the same use case.
Thanks,
Correct me if I'm wrong, when you run terraform init you are asked to name a storage account and container for the terraform state.
Can these also automatically be made with terraform?
Edit: I'm using Azure.
I usually split my terraform configurations into two parts.
One that creates a storage account with container, with a specific tag (tf=backend for example). The second one that creates all other resources. I share a backend.tfvars between the two, and in the second one, I get the storage account key using Azure CLI and the previously set tag (that way I don't have to get the key and pass it manually to my second script).
You could even migrate the state of the first terraform configuration once deployed, if you don't want to rely on a local state
Yes, absolutely. You would in general want an S3 bucket for each of your environments, although it's also possible to have a bucket shared across all environments and then set up access controls using bucket policies. Don't create this bucket as part of provisioning other resources, as their lifecycles will likely be different (you would want to retain the bucket for a long time and would be unlikely to want to destroy it).
What you do is you define this bucket in Terraform using local state first. After it is created, you add a remote backend pointing to this bucket.
terraform {
required_version = ">= 0.11.7"
backend "s3" {
bucket = "my-state-bucket"
key = "s3_state_bucket"
region = "us-west-2"
encrypt = "true"
}
}
After you run terraform init, Terraform will ask if you want to migrate the local state file to S3. Answer yes, and after this completes you can delete the local state file, as it's no longer used.
This approach allows you to break out of this chicken and egg situation and still manage all of your infrastructure as code, rather then creating it manually using web console or bash scripts.
When using Terraform to deploy our AWS infra, if we use the AWS console to redeploy an API Gateway deployment, it creates a new deployment ID. Afterwards when running terraform apply again, Terraform lists aws_api_gateway_stage.deployment_id as an in-place update item. Running terraform refresh does not update the state. It continues to show this as a delta. Am I missing something?
Terraform computes the delta based on what it already has saved on the backend and what there is on AWS at the moment of application. You can use Terraform to apply changes to AWS, but not the other way around, i.e. you cannot change your AWS configuration and expect Terraform to pull the changes into its state.