I am trying to use Terraformer ( https://github.com/GoogleCloudPlatform/terraformer ) to import some of our legacy into Terraform for ease of maintainability going forward. I keep running into a few issues though.
Current State - I am on Terraform version 1.1.7 with a remote backend configuration on S3. The desired end state is to use Terraformer to generate configurations for the legacy infrastructure, and include them as a separate module in the backend remote state.
Problem(s) -
Terraformer only seems to work with Terraform version 0.21.31, and the state files it generates also confides in this. I did some lookup and the standard suggestion here was to use https://github.com/tfutils/tfenv and have multiple Terraform versions. However it seems that TF 0.21.31 does not support provider plugin sources, which means I can't install the Hashicorp AWS provider plugin, without which, Terraformer can't work. So I am stuck in a deadlock here. Curious to know how others solved it?
Assuming I do somehow manage to solve the version issue, Terraformer seems to generate a local state file. Now, since my remote backend is on S3, how do I merge this local state file to the remote backend state file, so that Terraform realises that I am just planning to import some infra, and not re-create them?
Any help here would be much appreciated. Thanks in advance!
Please let me know if we need some more additional info for a better prognosis!
Related
Hi Terraform techies ,
I have a problem statement here. I used Terraform 0.13.5 to create my infrastructure. Due to some of the constraints I need to move back to 0.12.18. when I have make changes in infrastructure ,I see that ,the state files generated with tf 0.13.5 don't work with 0.12.8. is there a way I can backport the state files.
This is a process, as far as I know there is not a shortcut. You will need to do a state migration which can be tedious depending on the size of the state file.
Another option would be to import the infrastructure into the 0.12 state, or use data sources instead of migrating.
I've come across some AWS resources that were not created through my terraform configuration that I later realized I need to update. An example is cloudwatch logs where in my initial config (like lambda, db, etc...) didn't have any specification to create them. however, later if I want to set any config for the logs Im having trouble adding those resources to my config. I believe I need to do a terraform import for those resources but it essentially requires me to issue that command before the terraform apply.
This isn't really that clean if I have a process where I can do only one command (terraform apply).
Any suggestions to manage the terraform import as part of the config only? Like an import if not available.
Is there a way to extract the infrastructure details by using terraform
e.g get a list of Linux server's version, firewall policy, opened ports, software packages installed etc..
My aim is to generate a block of code to describe the current server setup, then I can use a check list if validate against the code. therefor security loopholes can be identified and fixed
Not sure if I completely understand your question. But, there is not such an "automated" way to extract all the details of your not-terraformed infrastructure. Nevertheless, there exists a terraform import command with which you can import your existing resource (here the docs) to your state file.
Btw, if you are using Oracle Cloud, the Resource Discovery could be an option.
I have a terraform state file with me. Can I use that to provision the infra instead of providing terraform config files(.tf)?
I basically need the functionality to support rollbacks. So, in case some failure happens while running some terraform command, I want to rollback to the previous state.
Rollbacks are not well-supported. You can do this in Terraform Enterprise or Terraform Cloud, but it is not trivial. Here is a link to the HashiCorp Support article that explains the process.
Basically, you will find the last known good state file, download it to your local system, change the backend, then move the state file. If you're really lucky, you can pull the state and then push the desired state. Be careful around the version of the CLI terraform and the version in Enterprise or Cloud.
please have a look on following 2 links
https://developers.cloudflare.com/terraform/tutorial/roll-back/
How to Rollback to Previous State in terraform
Both recommend to version control your .tf files.
Unfortunately no .tfstate mentioned.
So it seems the only way to go is wih .tf files and not with the .tfstate.
I usually run all my Terraform scripts through Bastion server and all my code including the tf statefile resides on the same server. There happened this incident where my machine accidentally went down (hard reboot) and somehow the root filesystem got corrupted. Now my statefile is gone but my resources still exist and are running. I don't want to again run terraform apply to recreate the whole environment with a downtime. What's the best way to recover from this mess and what can be done so that this doesn't get repeated in future.
I have already taken a look at terraform refresh and terraform import. But are there any better ways to do this ?
and all my code including the tf statefile resides on the same server.
As you don't have .backup file, I'm not sure if you can recover the statefile smoothly in terraform way, do let me know if you find a way :) . However you can take few step which will help you come out from situation like this.
The best practice is keep all your statefiles in some remote storage like S3 or Blob and configure your backend accordingly so that each time you destroy or create a new stack, it will always contact the statefile remotely.
On top of it, you can take the advantage of terraform workspace to avoid the mess of statefile in multi environment scenario. Also consider creating a plan for backtracking and versioning of previous deployments.
terraform plan -var-file "" -out "" -target=module.<blue/green>
what can be done so that this doesn't get repeated in future.
Terraform blue-green deployment is the answer to your question. We implemented this model quite a while and it's running smoothly. The whole idea is modularity and reusability, same templates is working for 5 different component with different architecture without any downtime(The core template remains same and variable files is different).
We are taking advantage of Terraform module. We have two module called blue and green, you can name anything. At any given point of time either blue or green will be taking traffic. If we have some changes to deploy we will bring the alternative stack based on state output( targeted module based on terraform state), auto validate it then move the traffic to the new stack and destroy the old one.
Here is an article you can keep as reference but this exactly doesn't reflect what we do nevertheless good to start with.
Please see this blog post, which, unfortunately, illustrates import being the only solution.
If you are still unable to recover the terraform state. You can create a blueprint of terraform configuration as well as state for a specific aws resources using terraforming But it requires some manual effort to edit the state for managing the resources back. You can have this state file, run terraform plan and compare its output with your infrastructure. It is good to have remote state especially using any object stores like aws s3 or key value store like consul. It has support for locking the state when multiple transactions happened at a same time. Backing up process is also quite simple.