Hi Terraform techies ,
I have a problem statement here. I used Terraform 0.13.5 to create my infrastructure. Due to some of the constraints I need to move back to 0.12.18. when I have make changes in infrastructure ,I see that ,the state files generated with tf 0.13.5 don't work with 0.12.8. is there a way I can backport the state files.
This is a process, as far as I know there is not a shortcut. You will need to do a state migration which can be tedious depending on the size of the state file.
Another option would be to import the infrastructure into the 0.12 state, or use data sources instead of migrating.
Related
I have recently started working on Terraform, have a question on terraform state mv and terraform import. As per the documentation, terraform state mv can be used when a resource name changes, and the updated name has to be added to the state file. And terraform import can be used to import the resources created outside of Terraform to a state file. My question is even when a resource name changes or code structure changes(using modules), we can still use terraform import to update the state file correct? Could anyone tell me, what is the real benefit of using terraform state mv command?
So the question really is this particular case:
I have renamed the TF resource / changed the structure of the resource
in IaC. Can I just re-import it into the new structure, instead of moving it?
Yes you can, but what will happen to the state? You'll be importing a resource you're already managing according to the TF state. The old resource that you've modified should still be managed, therefore you might run into issues where the TF operator will attempt to recreate it or even delete it. It will all depend on what state matches the reality in your cloud provider.
If you'd like to still import the updated, I'd go for terraform state rm & terraform import afterwards. This is sometimes required / an easy hack after big changes to a particular module / resource. It's also a good debugging experience, when you're not exactly sure about how does the cloud resource matches the TF code, as you're see state differences only for this newly imported resource.
One benefit of terraform state mv is useful if you need to refactor your code in or out of modules. I've used it quite a bit. I recommend backing up your state before making any changes. If you are using a remote state, you can always take a copy of it, disable your use of the remote state temporarily and then utilize the copy locally.
You can see the names of your state objects by using terraform state list.
The usage of terraform import is to add an existing thing to your state file, so it's tracked.
Terraform Import - Terraform is able to import existing infrastructure. This allows you take resources you've created by some other means and bring it under Terraform management.
Terraform State MV - It is less common situation where you wish to retain an existing remote object but track it as a different resource instance address in Terraform, such as if you have renamed a resource block or you have moved it into a different module in your configuration.
Use terraform import for all resources, created outside terraform
Use terraform state mv in the case, you want to restruct a already exisiting terraform resource.
I am using terraform state mv as soon as my projects needs to be restructed, e.g. become more complex, want to move to modules, etc.
Sometimes (even for older terraform projects), it could also be a good practice to import the resource again (with another name) and to a terraform state rm.
Does Terraform allow storing the state file in SVN? If this is not directly supported by Terraform, do any third party/ open source options exist?
When using Terraform it's not typical to store the Terraform state in the same version control repository as the configuration that drives it, because the expected workflow to use version control with Terraform is to review and commit the proposed changes first, and only then apply the changes to your real infrastructure from your main branch.
To understand why it might help to think about the relationship between an application's main code and its database. We don't typically store the main database for a web application in the version control repository along with the code, because the way we interact with the two are different: many developers can be concurrently working on and proposing changes to the application source code, and our version control system is often able to merge together those separate proposals to reduce collisions, but for the application's central database it's more common to use locks so that two writers don't try to change the same data (or interconnected data) at the same time.
In this way, Terraform state is roughly analogous to Terraform's "backend database". When using Terraform in a team setting then, you'd typically select one of the backends that stores state remotely and supports locking, and then anyone working with that particular Terraform configuration will find that Terraform will take out a lock before making any remote system modifications, hold that lock throughout its work, and then write the newly-updated state to the backend before releasing the lock.
Although you specifically asked about Subversion, my suggestions here are intended to apply to all version control systems. Version control is a good place to keep the source code for your Terraform modules, but it's not a good place to keep your Terraform state.
I am new to terraform. Can someone please explain
why do we need to save .tfstate file in local or remote storage,
when terraform apply always refreshes the state file with new infrastructure.
Thanks in advance.
The state file tracks the resources that Terraform is managing, whether it created them or imported them. Terraform's refresh only detects drift in managed resources and won't detect if you have created new resources outside of the state file.
If you lose the state you will end up with orphaned resources that are not being managed by Terraform. If, for some reason, you are okay with that or you have some other way of sharing state with other team members/CI and backing it up then you're fine.
Of course, using Terraform's remote state neatly solves those things so you should use it if you care about any of those things or think you might need to in the future (you probably will).
I will add a more developer-oriented perspective to help understand.
Think about you are using yarn or npm to do a NodeJS app, package.json is like your tf files, while yarn.lock or package-lock.json.
Dont take literally though, as terraform state file has physical underlying implications.
I have a terraform state file with me. Can I use that to provision the infra instead of providing terraform config files(.tf)?
I basically need the functionality to support rollbacks. So, in case some failure happens while running some terraform command, I want to rollback to the previous state.
Rollbacks are not well-supported. You can do this in Terraform Enterprise or Terraform Cloud, but it is not trivial. Here is a link to the HashiCorp Support article that explains the process.
Basically, you will find the last known good state file, download it to your local system, change the backend, then move the state file. If you're really lucky, you can pull the state and then push the desired state. Be careful around the version of the CLI terraform and the version in Enterprise or Cloud.
please have a look on following 2 links
https://developers.cloudflare.com/terraform/tutorial/roll-back/
How to Rollback to Previous State in terraform
Both recommend to version control your .tf files.
Unfortunately no .tfstate mentioned.
So it seems the only way to go is wih .tf files and not with the .tfstate.
I usually run all my Terraform scripts through Bastion server and all my code including the tf statefile resides on the same server. There happened this incident where my machine accidentally went down (hard reboot) and somehow the root filesystem got corrupted. Now my statefile is gone but my resources still exist and are running. I don't want to again run terraform apply to recreate the whole environment with a downtime. What's the best way to recover from this mess and what can be done so that this doesn't get repeated in future.
I have already taken a look at terraform refresh and terraform import. But are there any better ways to do this ?
and all my code including the tf statefile resides on the same server.
As you don't have .backup file, I'm not sure if you can recover the statefile smoothly in terraform way, do let me know if you find a way :) . However you can take few step which will help you come out from situation like this.
The best practice is keep all your statefiles in some remote storage like S3 or Blob and configure your backend accordingly so that each time you destroy or create a new stack, it will always contact the statefile remotely.
On top of it, you can take the advantage of terraform workspace to avoid the mess of statefile in multi environment scenario. Also consider creating a plan for backtracking and versioning of previous deployments.
terraform plan -var-file "" -out "" -target=module.<blue/green>
what can be done so that this doesn't get repeated in future.
Terraform blue-green deployment is the answer to your question. We implemented this model quite a while and it's running smoothly. The whole idea is modularity and reusability, same templates is working for 5 different component with different architecture without any downtime(The core template remains same and variable files is different).
We are taking advantage of Terraform module. We have two module called blue and green, you can name anything. At any given point of time either blue or green will be taking traffic. If we have some changes to deploy we will bring the alternative stack based on state output( targeted module based on terraform state), auto validate it then move the traffic to the new stack and destroy the old one.
Here is an article you can keep as reference but this exactly doesn't reflect what we do nevertheless good to start with.
Please see this blog post, which, unfortunately, illustrates import being the only solution.
If you are still unable to recover the terraform state. You can create a blueprint of terraform configuration as well as state for a specific aws resources using terraforming But it requires some manual effort to edit the state for managing the resources back. You can have this state file, run terraform plan and compare its output with your infrastructure. It is good to have remote state especially using any object stores like aws s3 or key value store like consul. It has support for locking the state when multiple transactions happened at a same time. Backing up process is also quite simple.