Terraform apply command fails when runs twice in Gitlab pipeline and resources were already created [duplicate] - gitlab

Question

This question already has an answer here:
Terraform destroy fails in CircleCI
(1 answer)
Closed 3 years ago.
I'm quite new with Terraform and I'm trying to replicate in my terraform configuration the stack I have already built for production (basically: Api gateway - Lambda - DynamoDB).
If I run terraform init, terraform plan and then terraform apply from my local host, everything is created as I want.
The problem arises when it comes to my Gitlab CI/CD pipeline, as Terraform complains about the existing resources (the first time runs properly, the second time complains and throws an error).
My Terraform steps in my .gitlab-ci.yml file:
plan:
stage: plan
image:
name: hashicorp/terraform:light
entrypoint:
- '/usr/bin/env'
- 'PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin'
script:
- cd terraform
- rm -rf .terraform
- terraform --version
- terraform init
- terraform plan
deploy:
stage: deploy
image:
name: hashicorp/terraform:light
entrypoint:
- '/usr/bin/env'
- 'PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin'
script:
- cd terraform
- terraform init
- terraform apply -auto-approve
dependencies:
- plan
when: manual
I see in my pipeline console the following error:
So after some Googling I saw that maybe the terraform import command could help.
Then added this import command to my .gitlab-ci.yml:
script:
- cd terraform
- terraform init
- terraform import aws_dynamodb_table.demo-dynamodb-table demo-dynamodb-table
- terraform apply -auto-approve
And the error in the Gitlab console was:
In the meantime I tried also this last change locally, and the error was:
So to summarize: I would need to know how to use Terraform in the right way to be able to run the apply command in my Gitlab CI/CD pipeline without conflicts with the resource that was created in the previous run of this same pipeline.

Answer 1

As others have stated, you need to store the Terraform state.
In my GitLab projects, I use a S3 bucket to store the Terraform state. But, have the CI pipeline fill in the key based on the GitLab project's path by setting the TF_CLI_ARGS_init environment variable.
terraform {
backend "s3" {
bucket = "bucket-name-here"
region = "us-west-2"
# key = $CI_PROJECT_PATH_SLUG
}
}
I also set the Terraform workspace based on the project. This can be modified to support branches. I also set the name variable to the project name, for use in the Terraform configuration. And, set input to false so that the CI job doesn't get hung up on user prompts.
variables:
TF_INPUT: "false"
TF_WORKSPACE: "$CI_PROJECT_NAME"
TF_VAR_name: "$CI_PROJECT_NAME"
TF_CLI_ARGS_init: "-upgrade=true"
For destroys, I also make sure to delete the workspace, so that there isn't stuff left over in the bucket.
.destroy:
extends: .terraform
stage: Cleanup
script:
- terraform init
- terraform destroy
-auto-approve
- export WORKSPACE=$TF_WORKSPACE
- export TF_WORKSPACE=default
- terraform workspace delete "$WORKSPACE"