Local TFstate does not migrate to remote location (Azure Blob) - azure

I am using Azure Blob to store my state. I follow these [steps] (https://github.com/hashicorp/terraform-cdk/blob/main/docs/working-with-cdk-for-terraform/remote-backend.md#migrating-local-state-storage-to-remote) the only difference is that I am using the AzurermBackend. The problem is when I do terraform init it does not migrate the existing state to the blob, it just create a new one in which there is no resources, so when i execute cdktf diff terraform says that it needs to create each resource that was already created in the local state. I checked the file the file is empty. I also tried with thr stack.addOveride that don't works too. Next thing I did is I execute the TF_LOG=DEBUG terraform init and got the following logs:
2021-12-20T16:00:03.228+0100 [DEBUG] Adding temp file log sink: /tmp/terraform-log769761292
2021-12-20T16:00:03.228+0100 [INFO] Terraform version: 1.0.9
2021-12-20T16:00:03.228+0100 [INFO] Go runtime version: go1.16.4
2021-12-20T16:00:03.228+0100 [INFO] CLI args: []string{"/usr/bin/terraform", "init"}
2021-12-20T16:00:03.228+0100 [DEBUG] Attempting to open CLI config file: /home/shurbeski/.terraformrc
2021-12-20T16:00:03.228+0100 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2021-12-20T16:00:03.228+0100 [DEBUG] ignoring non-existing provider search directory terraform.d/plugins
2021-12-20T16:00:03.228+0100 [DEBUG] ignoring non-existing provider search directory /home/shurbeski/.terraform.d/plugins
2021-12-20T16:00:03.228+0100 [DEBUG] ignoring non-existing provider search directory /home/shurbeski/.local/share/terraform/plugins
2021-12-20T16:00:03.228+0100 [DEBUG] ignoring non-existing provider search directory /usr/share/ubuntu/terraform/plugins
2021-12-20T16:00:03.228+0100 [DEBUG] ignoring non-existing provider search directory /usr/local/share/terraform/plugins
2021-12-20T16:00:03.228+0100 [DEBUG] ignoring non-existing provider search directory /usr/share/terraform/plugins
2021-12-20T16:00:03.228+0100 [DEBUG] ignoring non-existing provider search directory /var/lib/snapd/desktop/terraform/plugins
2021-12-20T16:00:03.228+0100 [INFO] CLI command args: []string{"init"}
Initializing the backend...
2021-12-20T16:00:03.229+0100 [DEBUG] New state was assigned lineage "2abdb28d-45b7-02a5-d5b1-851b3c446ef3"
2021-12-20T16:00:03.229+0100 [DEBUG] checking for provisioner in "."
2021-12-20T16:00:03.233+0100 [DEBUG] checking for provisioner in "/usr/bin"
2021-12-20T16:00:03.233+0100 [INFO] Failed to read plugin lock file .terraform/plugins/linux_amd64/lock.json: open .terraform/plugins/linux_amd64/lock.json: no such file or directory
2021-12-20T16:00:03.233+0100 [DEBUG] New state was assigned lineage "ea01857e-a1b7-080a-dda5-a5081c10f48b"
Actually it just creates a new state, so I tried TF_LOG=DEBUG terraform init -migrate-state and got the following logs:
2021-12-20T16:08:07.541+0100 [DEBUG] Adding temp file log sink: /tmp/terraform-log411077971
2021-12-20T16:08:07.541+0100 [INFO] Terraform version: 1.0.9
2021-12-20T16:08:07.541+0100 [INFO] Go runtime version: go1.16.4
2021-12-20T16:08:07.541+0100 [INFO] CLI args: []string{"/usr/bin/terraform", "init", "-migrate-state"}
2021-12-20T16:08:07.541+0100 [DEBUG] Attempting to open CLI config file: /home/shurbeski/.terraformrc
2021-12-20T16:08:07.541+0100 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2021-12-20T16:08:07.541+0100 [DEBUG] ignoring non-existing provider search directory terraform.d/plugins
2021-12-20T16:08:07.541+0100 [DEBUG] ignoring non-existing provider search directory /home/shurbeski/.terraform.d/plugins
2021-12-20T16:08:07.541+0100 [DEBUG] ignoring non-existing provider search directory /home/shurbeski/.local/share/terraform/plugins
2021-12-20T16:08:07.541+0100 [DEBUG] ignoring non-existing provider search directory /usr/share/ubuntu/terraform/plugins
2021-12-20T16:08:07.541+0100 [DEBUG] ignoring non-existing provider search directory /usr/local/share/terraform/plugins
2021-12-20T16:08:07.542+0100 [DEBUG] ignoring non-existing provider search directory /usr/share/terraform/plugins
2021-12-20T16:08:07.542+0100 [DEBUG] ignoring non-existing provider search directory /var/lib/snapd/desktop/terraform/plugins
2021-12-20T16:08:07.542+0100 [INFO] CLI command args: []string{"init", "-migrate-state"}
Initializing the backend...
2021-12-20T16:08:07.543+0100 [DEBUG] New state was assigned lineage "4af0afde-830e-1836-4bb8-4013609be0ad"
2021-12-20T16:08:07.970+0100 [DEBUG] checking for provisioner in "."
2021-12-20T16:08:07.974+0100 [DEBUG] checking for provisioner in "/usr/bin"
2021-12-20T16:08:07.974+0100 [INFO] Failed to read plugin lock file .terraform/plugins/linux_amd64/lock.json: open .terraform/plugins/linux_amd64/lock.json: no such file or directory
2021-12-20T16:08:07.975+0100 [DEBUG] New state was assigned lineage "472594f8-73dc-abe6-3691-5c7bddfb715e"
Even this didn't work.
The only thing that works if when I manually copy the tf state file and put it in the blob for the state, but i I do not like that.
Any ideas how would I get terraform to ask me if I want to migrate my pre-existing tfstate?
This is my code in the cdktf stack:
// new AzurermBackend(mystack, {
// storageAccountName: "cdkremotebackendtest",
// containerName: "test1",
// subscriptionId: "",
// key: "terraform.tfcdk-demo.tfstate",
// accessKey: "",
// });

You also need to specify a backend provider under main terraform config. If you don't specify it it will assume local so no migration. Something like this
terraform {
required_providers {
--------------------
}
backend "azurerm" {
resource_group_name = "cloud"
storage_account_name = "cdkremotebackendtest"
container_name = "test1"
key = "terraform.tfcdk-demo.tfstate"
}
}
More info on backends: https://www.terraform.io/language/settings/backends/configuration

Related

Terraform Plan Command Stuck

I have installed Terraform v1.1.7 and operating it using Visual Studio Code in Windows 10.
When I run any sample templates (using any provider like azure/local etc), the init works but the terraform plan and apply gets stuck for hours and does not give any output. I have tried all ways (re-install etc). I tried to looks at the trace logs and unable to find out the issue,
TERRAFORM CODE
resource "local_file" "demo" {
name = "1.txt"
content = "Hello World"
}
TERRAFORM CODE ERROR
2022-03-11T17:06:18.509+0530 [INFO] Terraform version: 1.1.7
2022-03-11T17:06:18.509+0530 [INFO] Go runtime version: go1.17.2
2022-03-11T17:06:18.509+0530 [INFO] CLI args: []string{"C:\\Users\\SushantaChakraborty\\Desktop\\tf\\terraform.exe", "plan"}
2022-03-11T17:06:18.512+0530 [TRACE] Stdout is not a terminal
2022-03-11T17:06:18.512+0530 [TRACE] Stderr is a terminal of width 155
2022-03-11T17:06:18.513+0530 [TRACE] Stdin is a terminal
2022-03-11T17:06:18.522+0530 [DEBUG] Attempting to open CLI config file: C:\Users\SushantaChakraborty\AppData\Roaming\terraform.rc
2022-03-11T17:06:18.523+0530 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2022-03-11T17:06:18.524+0530 [DEBUG] ignoring non-existing provider search directory terraform.d/plugins
2022-03-11T17:06:18.525+0530 [DEBUG] ignoring non-existing provider search directory C:\Users\SushantaChakraborty\AppData\Roaming\terraform.d\plugins
2022-03-11T17:06:18.526+0530 [DEBUG] ignoring non-existing provider search directory C:\Users\SushantaChakraborty\AppData\Roaming\HashiCorp\Terraform\plugins
2022-03-11T17:06:18.526+0530 [INFO] CLI command args: []string{"plan"}
2022-03-11T17:06:18.534+0530 [TRACE] Meta.Backend: no config given or present on disk, so returning nil config
2022-03-11T17:06:18.535+0530 [TRACE] Meta.Backend: backend has not previously been initialized in this working directory
2022-03-11T17:06:18.535+0530 [DEBUG] New state was assigned lineage "b61e8a78-e15f-ef87-8c9e-d78d607de4c3"
2022-03-11T17:06:18.536+0530 [TRACE] Meta.Backend: using default local state only (no backend configuration, and no existing initialized backend)
2022-03-11T17:06:18.536+0530 [TRACE] Meta.Backend: instantiated backend of type <nil>
2022-03-11T17:06:18.538+0530 [TRACE] providercache.fillMetaCache: scanning directory .terraform\providers
2022-03-11T17:06:18.548+0530 [TRACE] getproviders.SearchLocalDirectory: found registry.terraform.io/hashicorp/local v2.2.1 for windows_amd64 at .terraform\providers\registry.terraform.io\hashicorp\local\2.2.1\windows_amd64
2022-03-11T17:06:18.554+0530 [TRACE] providercache.fillMetaCache: including .terraform\providers\registry.terraform.io\hashicorp\local\2.2.1\windows_amd64
as a candidate package for registry.terraform.io/hashicorp/local 2.2.1
2022-03-11T17:06:19.953+0530 [DEBUG] checking for provisioner in "."
2022-03-11T17:06:19.954+0530 [DEBUG] checking for provisioner in "C:\\Users\\SushantaChakraborty\\Desktop\\tf"
2022-03-11T17:06:19.955+0530 [TRACE] Meta.Backend: backend <nil> does not support operations, so wrapping it in a local backend
2022-03-11T17:06:19.956+0530 [INFO] backend/local: starting Plan operation
2022-03-11T17:06:19.957+0530 [TRACE] backend/local: requesting state manager for workspace "default"
2022-03-11T17:06:19.957+0530 [TRACE] backend/local: state manager for workspace "default" will:
- read initial snapshot from terraform.tfstate
- write new snap`enter code here`shots to terraform.tfstate
- create any backup at terraform.tfstate.backup
2022-03-11T17:06:19.958+0530 [TRACE] backend/local: requesting state lock for workspace "default"
2022-03-11T17:06:25.543+0530 [TRACE] statemgr.Filesystem: preparing to manage state snapshots at terraform.tfstate
2022-03-11T17:06:25.545+0530 [TRACE] statemgr.Filesystem: no previously-stored snapshot exists
2022-03-11T17:06:25.546+0530 [TRACE] statemgr.Filesystem: locking terraform.tfstate using LockFileEx
2022-03-11T17:06:25.546+0530 [TRACE] statemgr.Filesystem: writing lock metadata to .terraform.tfstate.lock.info
2022-03-11T17:06:25.549+0530 [TRACE] backend/local: reading remote state for workspace "default"
2022-03-11T17:06:25.549+0530 [TRACE] statemgr.Filesystem: reading latest snapshot from terraform.tfstate
2022-03-11T17:06:25.550+0530 [TRACE] statemgr.Filesystem: snapshot file has nil snapshot, but that's okay
2022-03-11T17:06:25.550+0530 [TRACE] statemgr.Filesystem: read nil snapshot
2022-03-11T17:06:25.551+0530 [TRACE] backend/local: populating backend.LocalRun for current working directory
2022-03-11T17:06:25.553+0530 [TRACE] Config.VerifyDependencySelections: provider registry.terraform.io/hashicorp/local has 2.2.1 to satisfy ""
2022-03-11T17:06:25.554+0530 [TRACE] terraform.NewContext: starting
2022-03-11T17:06:25.554+0530 [TRACE] terraform.NewContext: complete
2022-03-11T17:06:25.554+0530 [TRACE] backend/local: requesting interactive input, if necessary
2022-03-11T17:06:25.555+0530 [TRACE] LoadSchemas: retrieving schema for provider type "registry.terraform.io/hashicorp/local"
2022-03-11T17:06:25.555+0530 [TRACE] terraform.contextPlugins: Initializing provider "registry.terraform.io/hashicorp/local" to read its schema
2022-03-11T17:06:25.556+0530 [DEBUG] created provider logger: level=trace
2022-03-11T17:06:25.556+0530 [INFO] provider: configuring client automatic mTLS
2022-03-11T17:06:25.596+0530 [DEBUG] provider: starting plugin: path=.terraform/providers/registry.terraform.io/hashicorp/local/2.2.1/windows_amd64/terraform-provider-local_v2.2.1_x5.exe args=[.terraform/providers/registry.terraform.io/hashicorp/local/2.2.1/windows_amd64/terraform-provider-local_v2.2.1_x5.exe]
2022-03-11T17:06:27.337+0530 [DEBUG] provider: plugin started: path=.terraform/providers/registry.terraform.io/hashicorp/local/2.2.1/windows_amd64/terraform-provider-local_v2.2.1_x5.exe pid=23380
2022-03-11T17:06:27.338+0530 [DEBUG] provider: waiting for RPC address: path=.terraform/providers/registry.terraform.io/hashicorp/local/2.2.1/windows_amd64/terraform-provider-local_v2.2.1_x5.exe
2022-03-11T17:06:30.099+0530 [INFO] provider.terraform-provider-local_v2.2.1_x5.exe: configuring server automatic mTLS: timestamp=2022-03-11T17:06:30.098+0530
2022-03-11T17:06:30.164+0530 [DEBUG] provider: using plugin: version=5
2022-03-11T17:06:30.167+0530 [DEBUG] provider.terraform-provider-local_v2.2.1_x5.exe: plugin address: address=127.0.0.1:10000 network=tcp timestamp=2022-03-11T17:06:30.164+0530
2022-03-11T17:06:30.242+0530 [TRACE] provider.stdio: waiting for stdio data
2022-03-11T17:06:30.242+0530 [TRACE] GRPCProvider: GetProviderSchema
2022-03-11T17:06:30.243+0530 [TRACE] provider.terraform-provider-local_v2.2.1_x5.exe: Received request: #caller=github.com/hashicorp/terraform-plugin-go#v0.5.0/tfprotov5/tf5server/server.go:329 tf_proto_version=5 #module=sdk.proto tf_provider_addr=provider tf_req_id=5a0134e9-eb48-03f5-a2ec-5fe6f4211828 tf_rpc=GetSchema timestamp=2022-03-11T17:06:30.243+0530
2022-03-11T17:06:30.244+0530 [TRACE] provider.terraform-provider-local_v2.2.1_x5.exe: Calling downstream: tf_provider_addr=provider #caller=github.com/hashicorp/terraform-plugin-go#v0.5.0/tfprotov5/tf5server/server.go:336 #module=sdk.proto tf_proto_version=5 tf_req_id=5a0134e9-eb48-03f5-a2ec-5fe6f4211828 tf_rpc=GetSchema timestamp=2022-03-11T17:06:30.243+0530
2022-03-11T17:06:30.244+0530 [TRACE] provider.terraform-provider-local_v2.2.1_x5.exe: Called downstream: #caller=github.com/hashicorp/terraform-plugin-go#v0.5.0/tfprotov5/tf5server/server.go:342 tf_proto_version=5 tf_provider_addr=provider #module=sdk.proto tf_req_id=5a0134e9-eb48-03f5-a2ec-5fe6f4211828 tf_rpc=GetSchema timestamp=2022-03-11T17:06:30.243+0530
2022-03-11T17:06:30.245+0530 [TRACE] GRPCProvider: Close
2022-03-11T17:06:30.245+0530 [TRACE] provider.terraform-provider-local_v2.2.1_x5.exe: Served request: tf_rpc=GetSchema #caller=github.com/hashicorp/terraform-plugin-go#v0.5.0/tfprotov5/tf5server/server.go:348 #module=sdk.proto tf_req_id=5a0134e9-eb48-03f5-a2ec-5fe6f4211828 tf_proto_version=5 tf_provider_addr=provider timestamp=2022-03-11T17:06:30.243+0530`

Error refreshing state: Failed to read remote state: unexpected EOF - Terraform State on GitLab

I'm try configuring GitLab to save terraform state.
I set my main.tf file as below:
terraform {
backend "http" {
address = "https://<myGitLab>/api/v4/projects/<ID-Project>/terraform/state/<Project-Name>"
lock_address = "https://<myGitLab>/api/v4/projects/<ID-Project>/terraform/state/<Project-Name>/lock"
unlock_address = "https://<myGitLab>/api/v4/projects/<ID-Project>/terraform/state/<Project-Name>/lock"
username = "myuser"
password = "<Token-ID>"
lock_method = "POST"
unlock_method = "DELETE"
retry_wait_min = 5
}
}
I can run only the first time, terraform init,terraform plan and terrafrom apply
The state is saved correctly on my GitLab project (under Infrastructure->Terraform).
But it I try to re-run terraform init I receive this error:
Initializing modules...
Initializing the backend...
2022/01/28 17:06:24 [DEBUG] GET https://<myGitLab>/api/v4/projects/<ID-Project>/terraform/state/<Project-Name>
Error refreshing state: Failed to read remote state: unexpected EOF
Note: my terraform files are not in the main root project, but in a subfolder:
/<Name-Project>/<terraform-folder>/main.tf
How can fix my problem?
UPDATE:
Now I receive this error when I run terraform plan:
Error: error loading state: Failed to read remote state: stream error: stream ID 3; INTERNAL_ERROR
These are the terraform logs:
2022-02-08T10:34:03.690Z [INFO] Terraform version: 1.1.4
2022-02-08T10:34:03.700Z [INFO] Go runtime version: go1.17.2
2022-02-08T10:34:03.701Z [INFO] CLI args: []string{"terraform", "plan", "-out=plan.out"}
2022-02-08T10:34:03.701Z [TRACE] Stdout is a terminal of width 120
2022-02-08T10:34:03.704Z [TRACE] Stderr is a terminal of width 120
2022-02-08T10:34:03.704Z [TRACE] Stdin is a terminal
2022-02-08T10:34:03.704Z [DEBUG] Attempting to open CLI config file: /root/.terraformrc
2022-02-08T10:34:03.705Z [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2022-02-08T10:34:03.706Z [DEBUG] ignoring non-existing provider search directory terraform.d/plugins
2022-02-08T10:34:03.706Z [DEBUG] ignoring non-existing provider search directory /root/.terraform.d/plugins
2022-02-08T10:34:03.706Z [DEBUG] ignoring non-existing provider search directory /root/.local/share/terraform/plugins
2022-02-08T10:34:03.708Z [DEBUG] ignoring non-existing provider search directory /usr/local/share/terraform/plugins
2022-02-08T10:34:03.710Z [DEBUG] ignoring non-existing provider search directory /usr/share/terraform/plugins
2022-02-08T10:34:03.711Z [INFO] CLI command args: []string{"plan", "-out=plan.out"}
2022-02-08T10:34:03.717Z [TRACE] Meta.Backend: built configuration for "http" backend with hash value 1000000006
2022-02-08T10:34:03.721Z [TRACE] Preserving existing state lineage "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
2022-02-08T10:34:03.721Z [TRACE] Preserving existing state lineage "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
2022-02-08T10:34:03.722Z [TRACE] Meta.Backend: working directory was previously initialized for "http" backend
2022-02-08T10:34:03.729Z [TRACE] Meta.Backend: using already-initialized, unchanged "http" backend configuration
2022-02-08T10:34:03.737Z [TRACE] Meta.Backend: instantiated backend of type *http.Backend
2022-02-08T10:34:03.741Z [TRACE] providercache.fillMetaCache: scanning directory .terraform/providers
2022-02-08T10:34:03.743Z [TRACE] getproviders.SearchLocalDirectory: found registry.terraform.io/tpretz/zabbix v0.15.0 for linux_amd64 at .terraform/providers/registry.terraform.io/tpretz/zabbix/0.15.0/linux_amd64
2022-02-08T10:34:03.744Z [TRACE] providercache.fillMetaCache: including .terraform/providers/registry.terraform.io/tpretz/zabbix/0.15.0/linux_amd64 as a candidate package for registry.terraform.io/tpretz/zabbix 0.15.0
2022-02-08T10:34:03.861Z [DEBUG] checking for provisioner in "."
2022-02-08T10:34:03.866Z [DEBUG] checking for provisioner in "/usr/bin"
2022-02-08T10:34:03.867Z [TRACE] Meta.Backend: backend *http.Backend does not support operations, so wrapping it in a local backend
2022-02-08T10:34:03.869Z [INFO] backend/local: starting Plan operation
2022-02-08T10:34:03.871Z [TRACE] backend/local: requesting state manager for workspace "default"
2022-02-08T10:34:03.872Z [TRACE] backend/local: requesting state lock for workspace "default"
2022-02-08T10:34:03.873Z [DEBUG] POST https://<myGitLab>/api/v4/projects/<ID-Project>/terraform/state/<Project-Name>/lock
2022-02-08T10:34:04.127Z [TRACE] backend/local: reading remote state for workspace "default"
2022-02-08T10:34:04.127Z [DEBUG] GET https://<myGitLab>/api/v4/projects/<ID-Project>/terraform/state/<Project-Name>
2022-02-08T10:34:04.475Z [DEBUG] DELETE https://<myGitLab>/api/v4/projects/<ID-Project>/terraform/state/<Project-Name>/lock

Why is terraform pulling a provider from Hashicorp I declare nowhere?

I am trying to test out some new code for a terraform provider and for reasons I don't understand, it seems to want to try and find a resource definition with hashicorp, even though it's specifically defined within this provider. Obviously I am missing something:
Code to [provider][1]:
This is how I build and install it:
christianb#unifi terraform-provider-artifactory % rm -fR .terraform
christianb#unifi terraform-provider-artifactory % go build && mkdir -p .terraform/plugins/registry.terraform.io/jfrog/artifactory/2.2.6-alpha/darwin_amd64 && mv terraform-provider-artifactory .terraform/plugins/registry.terraform.io/jfrog/artifactory/2.2.6-alpha/darwin_amd64
now running it
christianb#unifi terraform-provider-artifactory % TF_LOG=trace terraform init
2021/02/25 13:12:56 [INFO] Terraform version: 0.13.5
2021/02/25 13:12:56 [INFO] Go runtime version: go1.14.10
2021/02/25 13:12:56 [INFO] CLI args: []string{"/usr/local/bin/terraform", "init"}
2021/02/25 13:12:56 [DEBUG] Attempting to open CLI config file: /Users/christianb/.terraformrc
2021/02/25 13:12:56 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2021/02/25 13:12:56 [DEBUG] ignoring non-existing provider search directory terraform.d/plugins
2021/02/25 13:12:56 [DEBUG] ignoring non-existing provider search directory /Users/christianb/.terraform.d/plugins
2021/02/25 13:12:56 [DEBUG] ignoring non-existing provider search directory /Users/christianb/Library/Application Support/io.terraform/plugins
2021/02/25 13:12:56 [DEBUG] ignoring non-existing provider search directory /Library/Application Support/io.terraform/plugins
2021/02/25 13:12:56 [INFO] CLI command args: []string{"init"}
Initializing the backend...
2021/02/25 13:12:56 [TRACE] Meta.Backend: no config given or present on disk, so returning nil config
2021/02/25 13:12:56 [TRACE] Meta.Backend: backend has not previously been initialized in this working directory
2021/02/25 13:12:56 [DEBUG] New state was assigned lineage "b7f7e5b9-b88c-6195-aaaf-e38b3008b8e5"
2021/02/25 13:12:56 [TRACE] Meta.Backend: using default local state only (no backend configuration, and no existing initialized backend)
2021/02/25 13:12:56 [TRACE] Meta.Backend: instantiated backend of type <nil>
2021/02/25 13:12:56 [DEBUG] checking for provisioner in "."
2021/02/25 13:12:56 [DEBUG] checking for provisioner in "/usr/local/bin"
2021/02/25 13:12:56 [INFO] Failed to read plugin lock file .terraform/plugins/darwin_amd64/lock.json: open .terraform/plugins/darwin_amd64/lock.json: no such file or directory
2021/02/25 13:12:56 [TRACE] Meta.Backend: backend <nil> does not support operations, so wrapping it in a local backend
2021/02/25 13:12:56 [TRACE] backend/local: state manager for workspace "default" will:
- read initial snapshot from terraform.tfstate
- write new snapshots to terraform.tfstate
- create any backup at terraform.tfstate.backup
2021/02/25 13:12:56 [TRACE] statemgr.Filesystem: reading initial snapshot from terraform.tfstate
2021/02/25 13:12:56 [TRACE] statemgr.Filesystem: snapshot file has nil snapshot, but that's okay
2021/02/25 13:12:56 [TRACE] statemgr.Filesystem: read nil snapshot
2021/02/25 13:12:56 [TRACE] providercache.fillMetaCache: scanning directory .terraform/plugins
2021/02/25 13:12:56 [TRACE] getproviders.SearchLocalDirectory: .terraform/plugins is a symlink to .terraform/plugins
Initializing provider plugins...
2021/02/25 13:12:56 [TRACE] getproviders.SearchLocalDirectory: found registry.terraform.io/jfrog/artifactory v2.2.6-alpha for darwin_amd64 at .terraform/plugins/registry.terraform.io/jfrog/artifactory/2.2.6-alpha/darwin_amd64
2021/02/25 13:12:56 [TRACE] providercache.fillMetaCache: including .terraform/plugins/registry.terraform.io/jfrog/artifactory/2.2.6-alpha/darwin_amd64 as a candidate package for registry.terraform.io/jfrog/artifactory 2.2.6-alpha
2021/02/25 13:12:56 [DEBUG] Service discovery for registry.terraform.io at https://registry.terraform.io/.well-known/terraform.json
2021/02/25 13:12:56 [TRACE] HTTP client GET request to https://registry.terraform.io/.well-known/terraform.json
- Using previously-installed jfrog/artifactory v2.2.6-alpha
- Finding latest version of hashicorp/xray...
2021/02/25 13:12:56 [DEBUG] GET https://registry.terraform.io/v1/providers/hashicorp/xray/versions
2021/02/25 13:12:56 [TRACE] HTTP client GET request to https://registry.terraform.io/v1/providers/hashicorp/xray/versions
2021/02/25 13:12:57 [TRACE] providercache.fillMetaCache: using cached result from previous scan of .terraform/plugins
2021/02/25 13:12:57 [DEBUG] GET https://registry.terraform.io/v1/providers/-/xray/versions
2021/02/25 13:12:57 [TRACE] HTTP client GET request to https://registry.terraform.io/v1/providers/-/xray/versions
Error: Failed to install provider
Error while installing hashicorp/xray: provider registry registry.terraform.io
does not have a provider named registry.terraform.io/hashicorp/xray
terraform to run:
# Required for Terraform 0.13 and up (https://www.terraform.io/upgrade-guides/0-13.html)
terraform {
required_providers {
artifactory = {
source = "registry.terraform.io/jfrog/artifactory"
version = "2.2.6-alpha"
}
}
}
provider "artifactory" {
url = "http://localhost:8082/artifactory"
username = "admin"
password = "password"
}
resource "xray_policy" "test" {
name = "test-policy-name"
description = "test policy description"
type = "security"
rules {
name = "rule-name"
priority = 1
criteria {
min_severity = "High"
}
actions {
block_download {
unscanned = true
active = true
}
}
}
}
resource "xray_watch" "test" {
name = "test-watch-name"
description = "test watch description"
resources {
type = "all-repos"
name = "All Repositories"
}
assigned_policies {
name = xray_policy.test.name
type = "security"
}
watch_recipients = ["test#example.com"]
}
I figured out it's based on the name of the resource type - so, that it doesn't start with artifactory makes a difference. What I'd like to know is the logic behind this lookup
[1]: https://github.com/ryndaniels/terraform-provider-xray
Terraform v0.13 introduced the idea of third-party providers that belong to other namespaces that are not controlled directly by HashiCorp, but to maximize backward compatibility with modules that were written for Terraform v0.12 and earlier there is a fallback behavior where Terraform will assume that any provider requirement not explicitly declared is aiming to use one of the official providers which now live in the "hashicorp" namespace in the registry, because for Terraform v0.12 and earlier third-party providers were not automatically installable at all.
When writing modules for Terraform v0.13 or later you should include explicit provider requirements to specify the full source addresses for each of the providers your module uses, like this:
terraform {
required_providers {
artifactory = {
source = "jfrog/artifactory"
}
xray = {
source = "ryndaniels/xray"
}
}
}
Unfortunately the "xray" provider you are trying to use here hasn't, at the time of my writing this, been published in the Terraform Registry. The source address ryndaniels/xray I used above is the address this provider would take if the author were to publish it to the registry as-is, because the Terraform Registry uses a methodical naming scheme to generate provider source addresess from GitHub repository addresses.
But unless that provider is later published in the registry, you'll need to install it on your local system manually in order for Terraform to find it and use it. To achieve that, refer to Implied Local Mirror Directories to see which directories Terraform searches for plugins on your platform, and then you can create the necessary directory structure for Terraform to find the provider.
For the sake of an example I'm going to use the $HOME/.terraform.d/plugins/ prefix that Terraform supports on Linux, but note that the paths are different on Windows and macOS and so you will need to adjust to a different path prefix on those operating systems:
$HOME/.terraform.d/plugins/registry.terraform.io/ryndaniels/xray/0.0.1/linux_amd64/terraform-provider-xray
A few more notes about the above:
The registry.terraform.io/ryndaniels/xray part is what Terraform matches with the source you set in your provider_requirements block. registry.terraform.io is the default hostname for a source address that doesn't include a hostname, so ryndaniels/xray is short for registry.terraform.io/ryndaniels/xray.
This particular provider doesn't have any tagged releases and so there isn't a specific version number to use. Because Terraform expects all providers to have a version number, I arbitrarily chose 0.0.1 as the version number in the path above. This will only matter if you write an explicit version argument in your module to constrain which versions are allowed.
I used linux_amd64 as the target platform here, continuing my example of using Linux-style mirror paths. You can find out which platform is correct for you by running terraform version, which on recent Terraform versions includes the name of the platform that your Terraform CLI executable was built for:
Terraform v0.14.4
on linux_amd64
On startup Terraform will scan all of the implied local mirror directories for your current platform and will note any providers it finds there, and assume that you intend to install those from your local filesystem rather than from the registry. Therefore as long as the directory structure is correct as above, terraform init will see that you have ryndaniels/xray installed locally and will use the copy from there rather than asking the Terraform Registry which versions are available for that provider.
If the provider author publishes this provider to the Terraform Registry at a later date, and they do so without renaming the repository or moving it into a different GitHub account, it should appear in the registry as ryndaniels/xray and thus terraform init will be able to automatically install it from there without any further modifications to your modules.

Azure Pipelines : Terraform Apply fails when given a tfplan

I'm trying to use Terraform with Azure Pipelines. I use the 0.12.24 version of Terraform.
The steps are the basics :
Install Terraform 0.12.24,
Terraform 'init -reconfigure',
Terraform 'plan -out=$(Agent.TempDirectory)/my.tfplan)',
Terraform 'apply'
Everything goes smoothly until step 4. If I specify the tfplan file ($(Agent.TempDirectory)/my.tfplan), this step fails. If I don't, deployment ends successfully.
Here are the execution trace (generated by TF_LOG = TRACE) :
##[section]Starting: terraform apply
==============================================================================
Task : Terraform CLI
Description : Execute terraform cli commands
Version : 0.5.2
Author : Charles Zipp
Help :
==============================================================================
[command]C:\hostedtoolcache\windows\terraform\0.12.24\x64\terraform.exe version
2020/04/27 16:56:39 [INFO] Terraform version: 0.12.24
2020/04/27 16:56:39 [INFO] Go runtime version: go1.12.13
2020/04/27 16:56:39 [INFO] CLI args: []string{"C:\\hostedtoolcache\\windows\\terraform\\0.12.24\\x64\\terraform.exe", "version"}
2020/04/27 16:56:39 [DEBUG] Attempting to open CLI config file: C:\Users\VssAdministrator\AppData\Roaming\terraform.rc
2020/04/27 16:56:39 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2020/04/27 16:56:39 [INFO] CLI command args: []string{"version"}
Terraform v0.12.24
2020/04/27 16:56:39 [DEBUG] checking for provider in "."
2020/04/27 16:56:39 [DEBUG] checking for provider in "C:\\hostedtoolcache\\windows\\terraform\\0.12.24\\x64"
2020/04/27 16:56:39 [DEBUG] checking for provider in ".terraform\\plugins\\windows_amd64"
2020/04/27 16:56:39 [DEBUG] found provider "terraform-provider-azurerm_v2.4.0_x5.exe"
2020/04/27 16:56:39 [DEBUG] found valid plugin: "azurerm", "2.4.0", "D:\\a\\r1\\a\\Build\\drop\\terraform\\.terraform\\plugins\\windows_amd64\\terraform-provider-azurerm_v2.4.0_x5.exe"
+ provider.azurerm v2.4.0
[command]C:\hostedtoolcache\windows\terraform\0.12.24\x64\terraform.exe apply -auto-approve D:\a\_temp/my.tfplan
2020/04/27 16:56:40 [INFO] Terraform version: 0.12.24
2020/04/27 16:56:40 [INFO] Go runtime version: go1.12.13
2020/04/27 16:56:40 [INFO] CLI args: []string{"C:\\hostedtoolcache\\windows\\terraform\\0.12.24\\x64\\terraform.exe", "apply", "-auto-approve", "D:\\a\\_temp/my.tfplan"}
2020/04/27 16:56:40 [DEBUG] Attempting to open CLI config file: C:\Users\VssAdministrator\AppData\Roaming\terraform.rc
2020/04/27 16:56:40 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2020/04/27 16:56:40 [INFO] CLI command args: []string{"apply", "-auto-approve", "D:\\a\\_temp/my.tfplan"}
##[error]Terraform command 'apply' failed with exit code '1'.
##[section]Finishing: terraform apply
I've tried this with the two plugins available (the one from MSFT and the other from Charles Zipp).
Any question, input or suggestion is very much welcome.
Thank you for your time :)

Terraform Destroy Error when connected to TFE

I have created a workspace in Terraform Enterprise by running a terraform init && terraform plan locally with Terraform enterprise set up as my back end:
# Using a single workspace:
terraform {
backend "remote" {
hostname = "dep.app.example.io"
organization = "nnnn"
workspaces {
name = "create-workspace"
}
}
}
Terraform Apply works, and I can launch an ec2 via Terraform Enterprise with this code:
provider "aws" {
region = "${var.region}"
}
resource "aws_instance" "feature" {
count = 1
ami = "${var.ami}"
availability_zone = "${var.availability_zone}"
instance_type = "${var.instance_type}"
tags = {
Name = "${var.name_tag}"
}
}
Now when I run a terraform destroy, I get this error:
Error: error creating run: Invalid Attribute Infrastructure is
not destroyable
The configured "remote" backend encountered an unexpected
error. Sometimes this is caused by network connection problems,
in which case you could retry the command. If the issue
persists please open a support ticket to get help resolving the
problem.
What am I doing wrong here? I want to be able to run a terraform destroy that destroys the infrastructure my new Terraform enterprise workspace spins up.
EDIT: LOGS:
2019/04/03 09:11:54 [INFO] Terraform version: 0.11.11 ac4fff416318bf0915a0ab80e062a99ef3724334
2019/04/03 09:11:54 [INFO] Go runtime version: go1.11.1
2019/04/03 09:11:54 [INFO] CLI args: []string{"/usr/local/bin/terraform", "destroy"}
2019/04/03 09:11:54 [DEBUG] Attempting to open CLI config file: /Users/nlegorrec/.terraformrc
2019/04/03 09:11:54 Loading CLI configuration from /Users/nlegorrec/.terraformrc
2019/04/03 09:11:54 [INFO] CLI command args: []string{"destroy"}
2019/04/03 09:11:54 [TRACE] Preserving existing state lineage "f7abdc54-236c-c906-e701-049f3e2cc00c"
2019/04/03 09:11:54 [TRACE] Preserving existing state lineage "f7abdc54-236c-c906-e701-049f3e2cc00c"
2019/04/03 09:11:54 [DEBUG] Service discovery for dep.app.redbull.com at https://dep.app.redbull.com/.well-known/terraform.json
2019/04/03 09:11:56 [DEBUG] Retrieve version constraints for service tfe.v2 and product terraform
2019/04/03 09:11:57 [INFO] command: backend initialized: *remote.Remote
2019/04/03 09:11:57 [DEBUG] checking for provider in "."
2019/04/03 09:11:57 [DEBUG] checking for provider in "/usr/local/bin"
2019/04/03 09:11:57 [DEBUG] checking for provider in ".terraform/plugins/darwin_amd64"
2019/04/03 09:11:57 [DEBUG] found provider "terraform-provider-aws_v2.4.0_x4"
2019/04/03 09:11:57 [DEBUG] found valid plugin: "aws", "2.4.0", "/Users/nlegorrec/dev/Software Engineering/emp-kpi-tracker_web/dep/.terraform/plugins/darwin_amd64/terraform-provider-aws_v2.4.0_x4"
2019/04/03 09:11:57 [DEBUG] checking for provisioner in "."
2019/04/03 09:11:57 [DEBUG] checking for provisioner in "/usr/local/bin"
2019/04/03 09:11:57 [DEBUG] checking for provisioner in ".terraform/plugins/darwin_amd64"
2019/04/03 09:11:57 [INFO] backend/remote: starting Apply operation
2019/04/03 09:12:00 [DEBUG] plugin: waiting for all plugin processes to complete...
Error: error creating run: Invalid Attribute Infrastructure is not destroyable
The configured "remote" backend encountered an unexpected error. Sometimes
this is caused by network connection problems, in which case you could retry
the command. If the issue persists please open a support ticket to get help
resolving the problem.
Even though its a bit late hopefully this answer can help others in the future.
When using Terraform Enterprise or Terraform Cloud, you need to ensure that you are following their guidance on Destruction and Deletion from within the Workspace
Documentation for this is located here
To queue the destruction of infrastructure that is managed by a workspace you need to ensure that within the Variables of the workspace that you have assigned a variable with the name CONFIRM_DESTROY with a value of 1
Importantly, any changes to the workspace require admin privleges
Once you have completed that you should be able to use the CLI Workflow as you would locally in Terraform.

Resources