I have installed Terraform v1.1.7 and operating it using Visual Studio Code in Windows 10.
When I run any sample templates (using any provider like azure/local etc), the init works but the terraform plan and apply gets stuck for hours and does not give any output. I have tried all ways (re-install etc). I tried to looks at the trace logs and unable to find out the issue,
TERRAFORM CODE
resource "local_file" "demo" {
name = "1.txt"
content = "Hello World"
}
TERRAFORM CODE ERROR
2022-03-11T17:06:18.509+0530 [INFO] Terraform version: 1.1.7
2022-03-11T17:06:18.509+0530 [INFO] Go runtime version: go1.17.2
2022-03-11T17:06:18.509+0530 [INFO] CLI args: []string{"C:\\Users\\SushantaChakraborty\\Desktop\\tf\\terraform.exe", "plan"}
2022-03-11T17:06:18.512+0530 [TRACE] Stdout is not a terminal
2022-03-11T17:06:18.512+0530 [TRACE] Stderr is a terminal of width 155
2022-03-11T17:06:18.513+0530 [TRACE] Stdin is a terminal
2022-03-11T17:06:18.522+0530 [DEBUG] Attempting to open CLI config file: C:\Users\SushantaChakraborty\AppData\Roaming\terraform.rc
2022-03-11T17:06:18.523+0530 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2022-03-11T17:06:18.524+0530 [DEBUG] ignoring non-existing provider search directory terraform.d/plugins
2022-03-11T17:06:18.525+0530 [DEBUG] ignoring non-existing provider search directory C:\Users\SushantaChakraborty\AppData\Roaming\terraform.d\plugins
2022-03-11T17:06:18.526+0530 [DEBUG] ignoring non-existing provider search directory C:\Users\SushantaChakraborty\AppData\Roaming\HashiCorp\Terraform\plugins
2022-03-11T17:06:18.526+0530 [INFO] CLI command args: []string{"plan"}
2022-03-11T17:06:18.534+0530 [TRACE] Meta.Backend: no config given or present on disk, so returning nil config
2022-03-11T17:06:18.535+0530 [TRACE] Meta.Backend: backend has not previously been initialized in this working directory
2022-03-11T17:06:18.535+0530 [DEBUG] New state was assigned lineage "b61e8a78-e15f-ef87-8c9e-d78d607de4c3"
2022-03-11T17:06:18.536+0530 [TRACE] Meta.Backend: using default local state only (no backend configuration, and no existing initialized backend)
2022-03-11T17:06:18.536+0530 [TRACE] Meta.Backend: instantiated backend of type <nil>
2022-03-11T17:06:18.538+0530 [TRACE] providercache.fillMetaCache: scanning directory .terraform\providers
2022-03-11T17:06:18.548+0530 [TRACE] getproviders.SearchLocalDirectory: found registry.terraform.io/hashicorp/local v2.2.1 for windows_amd64 at .terraform\providers\registry.terraform.io\hashicorp\local\2.2.1\windows_amd64
2022-03-11T17:06:18.554+0530 [TRACE] providercache.fillMetaCache: including .terraform\providers\registry.terraform.io\hashicorp\local\2.2.1\windows_amd64
as a candidate package for registry.terraform.io/hashicorp/local 2.2.1
2022-03-11T17:06:19.953+0530 [DEBUG] checking for provisioner in "."
2022-03-11T17:06:19.954+0530 [DEBUG] checking for provisioner in "C:\\Users\\SushantaChakraborty\\Desktop\\tf"
2022-03-11T17:06:19.955+0530 [TRACE] Meta.Backend: backend <nil> does not support operations, so wrapping it in a local backend
2022-03-11T17:06:19.956+0530 [INFO] backend/local: starting Plan operation
2022-03-11T17:06:19.957+0530 [TRACE] backend/local: requesting state manager for workspace "default"
2022-03-11T17:06:19.957+0530 [TRACE] backend/local: state manager for workspace "default" will:
- read initial snapshot from terraform.tfstate
- write new snap`enter code here`shots to terraform.tfstate
- create any backup at terraform.tfstate.backup
2022-03-11T17:06:19.958+0530 [TRACE] backend/local: requesting state lock for workspace "default"
2022-03-11T17:06:25.543+0530 [TRACE] statemgr.Filesystem: preparing to manage state snapshots at terraform.tfstate
2022-03-11T17:06:25.545+0530 [TRACE] statemgr.Filesystem: no previously-stored snapshot exists
2022-03-11T17:06:25.546+0530 [TRACE] statemgr.Filesystem: locking terraform.tfstate using LockFileEx
2022-03-11T17:06:25.546+0530 [TRACE] statemgr.Filesystem: writing lock metadata to .terraform.tfstate.lock.info
2022-03-11T17:06:25.549+0530 [TRACE] backend/local: reading remote state for workspace "default"
2022-03-11T17:06:25.549+0530 [TRACE] statemgr.Filesystem: reading latest snapshot from terraform.tfstate
2022-03-11T17:06:25.550+0530 [TRACE] statemgr.Filesystem: snapshot file has nil snapshot, but that's okay
2022-03-11T17:06:25.550+0530 [TRACE] statemgr.Filesystem: read nil snapshot
2022-03-11T17:06:25.551+0530 [TRACE] backend/local: populating backend.LocalRun for current working directory
2022-03-11T17:06:25.553+0530 [TRACE] Config.VerifyDependencySelections: provider registry.terraform.io/hashicorp/local has 2.2.1 to satisfy ""
2022-03-11T17:06:25.554+0530 [TRACE] terraform.NewContext: starting
2022-03-11T17:06:25.554+0530 [TRACE] terraform.NewContext: complete
2022-03-11T17:06:25.554+0530 [TRACE] backend/local: requesting interactive input, if necessary
2022-03-11T17:06:25.555+0530 [TRACE] LoadSchemas: retrieving schema for provider type "registry.terraform.io/hashicorp/local"
2022-03-11T17:06:25.555+0530 [TRACE] terraform.contextPlugins: Initializing provider "registry.terraform.io/hashicorp/local" to read its schema
2022-03-11T17:06:25.556+0530 [DEBUG] created provider logger: level=trace
2022-03-11T17:06:25.556+0530 [INFO] provider: configuring client automatic mTLS
2022-03-11T17:06:25.596+0530 [DEBUG] provider: starting plugin: path=.terraform/providers/registry.terraform.io/hashicorp/local/2.2.1/windows_amd64/terraform-provider-local_v2.2.1_x5.exe args=[.terraform/providers/registry.terraform.io/hashicorp/local/2.2.1/windows_amd64/terraform-provider-local_v2.2.1_x5.exe]
2022-03-11T17:06:27.337+0530 [DEBUG] provider: plugin started: path=.terraform/providers/registry.terraform.io/hashicorp/local/2.2.1/windows_amd64/terraform-provider-local_v2.2.1_x5.exe pid=23380
2022-03-11T17:06:27.338+0530 [DEBUG] provider: waiting for RPC address: path=.terraform/providers/registry.terraform.io/hashicorp/local/2.2.1/windows_amd64/terraform-provider-local_v2.2.1_x5.exe
2022-03-11T17:06:30.099+0530 [INFO] provider.terraform-provider-local_v2.2.1_x5.exe: configuring server automatic mTLS: timestamp=2022-03-11T17:06:30.098+0530
2022-03-11T17:06:30.164+0530 [DEBUG] provider: using plugin: version=5
2022-03-11T17:06:30.167+0530 [DEBUG] provider.terraform-provider-local_v2.2.1_x5.exe: plugin address: address=127.0.0.1:10000 network=tcp timestamp=2022-03-11T17:06:30.164+0530
2022-03-11T17:06:30.242+0530 [TRACE] provider.stdio: waiting for stdio data
2022-03-11T17:06:30.242+0530 [TRACE] GRPCProvider: GetProviderSchema
2022-03-11T17:06:30.243+0530 [TRACE] provider.terraform-provider-local_v2.2.1_x5.exe: Received request: #caller=github.com/hashicorp/terraform-plugin-go#v0.5.0/tfprotov5/tf5server/server.go:329 tf_proto_version=5 #module=sdk.proto tf_provider_addr=provider tf_req_id=5a0134e9-eb48-03f5-a2ec-5fe6f4211828 tf_rpc=GetSchema timestamp=2022-03-11T17:06:30.243+0530
2022-03-11T17:06:30.244+0530 [TRACE] provider.terraform-provider-local_v2.2.1_x5.exe: Calling downstream: tf_provider_addr=provider #caller=github.com/hashicorp/terraform-plugin-go#v0.5.0/tfprotov5/tf5server/server.go:336 #module=sdk.proto tf_proto_version=5 tf_req_id=5a0134e9-eb48-03f5-a2ec-5fe6f4211828 tf_rpc=GetSchema timestamp=2022-03-11T17:06:30.243+0530
2022-03-11T17:06:30.244+0530 [TRACE] provider.terraform-provider-local_v2.2.1_x5.exe: Called downstream: #caller=github.com/hashicorp/terraform-plugin-go#v0.5.0/tfprotov5/tf5server/server.go:342 tf_proto_version=5 tf_provider_addr=provider #module=sdk.proto tf_req_id=5a0134e9-eb48-03f5-a2ec-5fe6f4211828 tf_rpc=GetSchema timestamp=2022-03-11T17:06:30.243+0530
2022-03-11T17:06:30.245+0530 [TRACE] GRPCProvider: Close
2022-03-11T17:06:30.245+0530 [TRACE] provider.terraform-provider-local_v2.2.1_x5.exe: Served request: tf_rpc=GetSchema #caller=github.com/hashicorp/terraform-plugin-go#v0.5.0/tfprotov5/tf5server/server.go:348 #module=sdk.proto tf_req_id=5a0134e9-eb48-03f5-a2ec-5fe6f4211828 tf_proto_version=5 tf_provider_addr=provider timestamp=2022-03-11T17:06:30.243+0530`
Related
I'm try configuring GitLab to save terraform state.
I set my main.tf file as below:
terraform {
backend "http" {
address = "https://<myGitLab>/api/v4/projects/<ID-Project>/terraform/state/<Project-Name>"
lock_address = "https://<myGitLab>/api/v4/projects/<ID-Project>/terraform/state/<Project-Name>/lock"
unlock_address = "https://<myGitLab>/api/v4/projects/<ID-Project>/terraform/state/<Project-Name>/lock"
username = "myuser"
password = "<Token-ID>"
lock_method = "POST"
unlock_method = "DELETE"
retry_wait_min = 5
}
}
I can run only the first time, terraform init,terraform plan and terrafrom apply
The state is saved correctly on my GitLab project (under Infrastructure->Terraform).
But it I try to re-run terraform init I receive this error:
Initializing modules...
Initializing the backend...
2022/01/28 17:06:24 [DEBUG] GET https://<myGitLab>/api/v4/projects/<ID-Project>/terraform/state/<Project-Name>
Error refreshing state: Failed to read remote state: unexpected EOF
Note: my terraform files are not in the main root project, but in a subfolder:
/<Name-Project>/<terraform-folder>/main.tf
How can fix my problem?
UPDATE:
Now I receive this error when I run terraform plan:
Error: error loading state: Failed to read remote state: stream error: stream ID 3; INTERNAL_ERROR
These are the terraform logs:
2022-02-08T10:34:03.690Z [INFO] Terraform version: 1.1.4
2022-02-08T10:34:03.700Z [INFO] Go runtime version: go1.17.2
2022-02-08T10:34:03.701Z [INFO] CLI args: []string{"terraform", "plan", "-out=plan.out"}
2022-02-08T10:34:03.701Z [TRACE] Stdout is a terminal of width 120
2022-02-08T10:34:03.704Z [TRACE] Stderr is a terminal of width 120
2022-02-08T10:34:03.704Z [TRACE] Stdin is a terminal
2022-02-08T10:34:03.704Z [DEBUG] Attempting to open CLI config file: /root/.terraformrc
2022-02-08T10:34:03.705Z [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2022-02-08T10:34:03.706Z [DEBUG] ignoring non-existing provider search directory terraform.d/plugins
2022-02-08T10:34:03.706Z [DEBUG] ignoring non-existing provider search directory /root/.terraform.d/plugins
2022-02-08T10:34:03.706Z [DEBUG] ignoring non-existing provider search directory /root/.local/share/terraform/plugins
2022-02-08T10:34:03.708Z [DEBUG] ignoring non-existing provider search directory /usr/local/share/terraform/plugins
2022-02-08T10:34:03.710Z [DEBUG] ignoring non-existing provider search directory /usr/share/terraform/plugins
2022-02-08T10:34:03.711Z [INFO] CLI command args: []string{"plan", "-out=plan.out"}
2022-02-08T10:34:03.717Z [TRACE] Meta.Backend: built configuration for "http" backend with hash value 1000000006
2022-02-08T10:34:03.721Z [TRACE] Preserving existing state lineage "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
2022-02-08T10:34:03.721Z [TRACE] Preserving existing state lineage "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
2022-02-08T10:34:03.722Z [TRACE] Meta.Backend: working directory was previously initialized for "http" backend
2022-02-08T10:34:03.729Z [TRACE] Meta.Backend: using already-initialized, unchanged "http" backend configuration
2022-02-08T10:34:03.737Z [TRACE] Meta.Backend: instantiated backend of type *http.Backend
2022-02-08T10:34:03.741Z [TRACE] providercache.fillMetaCache: scanning directory .terraform/providers
2022-02-08T10:34:03.743Z [TRACE] getproviders.SearchLocalDirectory: found registry.terraform.io/tpretz/zabbix v0.15.0 for linux_amd64 at .terraform/providers/registry.terraform.io/tpretz/zabbix/0.15.0/linux_amd64
2022-02-08T10:34:03.744Z [TRACE] providercache.fillMetaCache: including .terraform/providers/registry.terraform.io/tpretz/zabbix/0.15.0/linux_amd64 as a candidate package for registry.terraform.io/tpretz/zabbix 0.15.0
2022-02-08T10:34:03.861Z [DEBUG] checking for provisioner in "."
2022-02-08T10:34:03.866Z [DEBUG] checking for provisioner in "/usr/bin"
2022-02-08T10:34:03.867Z [TRACE] Meta.Backend: backend *http.Backend does not support operations, so wrapping it in a local backend
2022-02-08T10:34:03.869Z [INFO] backend/local: starting Plan operation
2022-02-08T10:34:03.871Z [TRACE] backend/local: requesting state manager for workspace "default"
2022-02-08T10:34:03.872Z [TRACE] backend/local: requesting state lock for workspace "default"
2022-02-08T10:34:03.873Z [DEBUG] POST https://<myGitLab>/api/v4/projects/<ID-Project>/terraform/state/<Project-Name>/lock
2022-02-08T10:34:04.127Z [TRACE] backend/local: reading remote state for workspace "default"
2022-02-08T10:34:04.127Z [DEBUG] GET https://<myGitLab>/api/v4/projects/<ID-Project>/terraform/state/<Project-Name>
2022-02-08T10:34:04.475Z [DEBUG] DELETE https://<myGitLab>/api/v4/projects/<ID-Project>/terraform/state/<Project-Name>/lock
I am using Azure Blob to store my state. I follow these [steps] (https://github.com/hashicorp/terraform-cdk/blob/main/docs/working-with-cdk-for-terraform/remote-backend.md#migrating-local-state-storage-to-remote) the only difference is that I am using the AzurermBackend. The problem is when I do terraform init it does not migrate the existing state to the blob, it just create a new one in which there is no resources, so when i execute cdktf diff terraform says that it needs to create each resource that was already created in the local state. I checked the file the file is empty. I also tried with thr stack.addOveride that don't works too. Next thing I did is I execute the TF_LOG=DEBUG terraform init and got the following logs:
2021-12-20T16:00:03.228+0100 [DEBUG] Adding temp file log sink: /tmp/terraform-log769761292
2021-12-20T16:00:03.228+0100 [INFO] Terraform version: 1.0.9
2021-12-20T16:00:03.228+0100 [INFO] Go runtime version: go1.16.4
2021-12-20T16:00:03.228+0100 [INFO] CLI args: []string{"/usr/bin/terraform", "init"}
2021-12-20T16:00:03.228+0100 [DEBUG] Attempting to open CLI config file: /home/shurbeski/.terraformrc
2021-12-20T16:00:03.228+0100 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2021-12-20T16:00:03.228+0100 [DEBUG] ignoring non-existing provider search directory terraform.d/plugins
2021-12-20T16:00:03.228+0100 [DEBUG] ignoring non-existing provider search directory /home/shurbeski/.terraform.d/plugins
2021-12-20T16:00:03.228+0100 [DEBUG] ignoring non-existing provider search directory /home/shurbeski/.local/share/terraform/plugins
2021-12-20T16:00:03.228+0100 [DEBUG] ignoring non-existing provider search directory /usr/share/ubuntu/terraform/plugins
2021-12-20T16:00:03.228+0100 [DEBUG] ignoring non-existing provider search directory /usr/local/share/terraform/plugins
2021-12-20T16:00:03.228+0100 [DEBUG] ignoring non-existing provider search directory /usr/share/terraform/plugins
2021-12-20T16:00:03.228+0100 [DEBUG] ignoring non-existing provider search directory /var/lib/snapd/desktop/terraform/plugins
2021-12-20T16:00:03.228+0100 [INFO] CLI command args: []string{"init"}
Initializing the backend...
2021-12-20T16:00:03.229+0100 [DEBUG] New state was assigned lineage "2abdb28d-45b7-02a5-d5b1-851b3c446ef3"
2021-12-20T16:00:03.229+0100 [DEBUG] checking for provisioner in "."
2021-12-20T16:00:03.233+0100 [DEBUG] checking for provisioner in "/usr/bin"
2021-12-20T16:00:03.233+0100 [INFO] Failed to read plugin lock file .terraform/plugins/linux_amd64/lock.json: open .terraform/plugins/linux_amd64/lock.json: no such file or directory
2021-12-20T16:00:03.233+0100 [DEBUG] New state was assigned lineage "ea01857e-a1b7-080a-dda5-a5081c10f48b"
Actually it just creates a new state, so I tried TF_LOG=DEBUG terraform init -migrate-state and got the following logs:
2021-12-20T16:08:07.541+0100 [DEBUG] Adding temp file log sink: /tmp/terraform-log411077971
2021-12-20T16:08:07.541+0100 [INFO] Terraform version: 1.0.9
2021-12-20T16:08:07.541+0100 [INFO] Go runtime version: go1.16.4
2021-12-20T16:08:07.541+0100 [INFO] CLI args: []string{"/usr/bin/terraform", "init", "-migrate-state"}
2021-12-20T16:08:07.541+0100 [DEBUG] Attempting to open CLI config file: /home/shurbeski/.terraformrc
2021-12-20T16:08:07.541+0100 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2021-12-20T16:08:07.541+0100 [DEBUG] ignoring non-existing provider search directory terraform.d/plugins
2021-12-20T16:08:07.541+0100 [DEBUG] ignoring non-existing provider search directory /home/shurbeski/.terraform.d/plugins
2021-12-20T16:08:07.541+0100 [DEBUG] ignoring non-existing provider search directory /home/shurbeski/.local/share/terraform/plugins
2021-12-20T16:08:07.541+0100 [DEBUG] ignoring non-existing provider search directory /usr/share/ubuntu/terraform/plugins
2021-12-20T16:08:07.541+0100 [DEBUG] ignoring non-existing provider search directory /usr/local/share/terraform/plugins
2021-12-20T16:08:07.542+0100 [DEBUG] ignoring non-existing provider search directory /usr/share/terraform/plugins
2021-12-20T16:08:07.542+0100 [DEBUG] ignoring non-existing provider search directory /var/lib/snapd/desktop/terraform/plugins
2021-12-20T16:08:07.542+0100 [INFO] CLI command args: []string{"init", "-migrate-state"}
Initializing the backend...
2021-12-20T16:08:07.543+0100 [DEBUG] New state was assigned lineage "4af0afde-830e-1836-4bb8-4013609be0ad"
2021-12-20T16:08:07.970+0100 [DEBUG] checking for provisioner in "."
2021-12-20T16:08:07.974+0100 [DEBUG] checking for provisioner in "/usr/bin"
2021-12-20T16:08:07.974+0100 [INFO] Failed to read plugin lock file .terraform/plugins/linux_amd64/lock.json: open .terraform/plugins/linux_amd64/lock.json: no such file or directory
2021-12-20T16:08:07.975+0100 [DEBUG] New state was assigned lineage "472594f8-73dc-abe6-3691-5c7bddfb715e"
Even this didn't work.
The only thing that works if when I manually copy the tf state file and put it in the blob for the state, but i I do not like that.
Any ideas how would I get terraform to ask me if I want to migrate my pre-existing tfstate?
This is my code in the cdktf stack:
// new AzurermBackend(mystack, {
// storageAccountName: "cdkremotebackendtest",
// containerName: "test1",
// subscriptionId: "",
// key: "terraform.tfcdk-demo.tfstate",
// accessKey: "",
// });
You also need to specify a backend provider under main terraform config. If you don't specify it it will assume local so no migration. Something like this
terraform {
required_providers {
--------------------
}
backend "azurerm" {
resource_group_name = "cloud"
storage_account_name = "cdkremotebackendtest"
container_name = "test1"
key = "terraform.tfcdk-demo.tfstate"
}
}
More info on backends: https://www.terraform.io/language/settings/backends/configuration
I am trying to test out some new code for a terraform provider and for reasons I don't understand, it seems to want to try and find a resource definition with hashicorp, even though it's specifically defined within this provider. Obviously I am missing something:
Code to [provider][1]:
This is how I build and install it:
christianb#unifi terraform-provider-artifactory % rm -fR .terraform
christianb#unifi terraform-provider-artifactory % go build && mkdir -p .terraform/plugins/registry.terraform.io/jfrog/artifactory/2.2.6-alpha/darwin_amd64 && mv terraform-provider-artifactory .terraform/plugins/registry.terraform.io/jfrog/artifactory/2.2.6-alpha/darwin_amd64
now running it
christianb#unifi terraform-provider-artifactory % TF_LOG=trace terraform init
2021/02/25 13:12:56 [INFO] Terraform version: 0.13.5
2021/02/25 13:12:56 [INFO] Go runtime version: go1.14.10
2021/02/25 13:12:56 [INFO] CLI args: []string{"/usr/local/bin/terraform", "init"}
2021/02/25 13:12:56 [DEBUG] Attempting to open CLI config file: /Users/christianb/.terraformrc
2021/02/25 13:12:56 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2021/02/25 13:12:56 [DEBUG] ignoring non-existing provider search directory terraform.d/plugins
2021/02/25 13:12:56 [DEBUG] ignoring non-existing provider search directory /Users/christianb/.terraform.d/plugins
2021/02/25 13:12:56 [DEBUG] ignoring non-existing provider search directory /Users/christianb/Library/Application Support/io.terraform/plugins
2021/02/25 13:12:56 [DEBUG] ignoring non-existing provider search directory /Library/Application Support/io.terraform/plugins
2021/02/25 13:12:56 [INFO] CLI command args: []string{"init"}
Initializing the backend...
2021/02/25 13:12:56 [TRACE] Meta.Backend: no config given or present on disk, so returning nil config
2021/02/25 13:12:56 [TRACE] Meta.Backend: backend has not previously been initialized in this working directory
2021/02/25 13:12:56 [DEBUG] New state was assigned lineage "b7f7e5b9-b88c-6195-aaaf-e38b3008b8e5"
2021/02/25 13:12:56 [TRACE] Meta.Backend: using default local state only (no backend configuration, and no existing initialized backend)
2021/02/25 13:12:56 [TRACE] Meta.Backend: instantiated backend of type <nil>
2021/02/25 13:12:56 [DEBUG] checking for provisioner in "."
2021/02/25 13:12:56 [DEBUG] checking for provisioner in "/usr/local/bin"
2021/02/25 13:12:56 [INFO] Failed to read plugin lock file .terraform/plugins/darwin_amd64/lock.json: open .terraform/plugins/darwin_amd64/lock.json: no such file or directory
2021/02/25 13:12:56 [TRACE] Meta.Backend: backend <nil> does not support operations, so wrapping it in a local backend
2021/02/25 13:12:56 [TRACE] backend/local: state manager for workspace "default" will:
- read initial snapshot from terraform.tfstate
- write new snapshots to terraform.tfstate
- create any backup at terraform.tfstate.backup
2021/02/25 13:12:56 [TRACE] statemgr.Filesystem: reading initial snapshot from terraform.tfstate
2021/02/25 13:12:56 [TRACE] statemgr.Filesystem: snapshot file has nil snapshot, but that's okay
2021/02/25 13:12:56 [TRACE] statemgr.Filesystem: read nil snapshot
2021/02/25 13:12:56 [TRACE] providercache.fillMetaCache: scanning directory .terraform/plugins
2021/02/25 13:12:56 [TRACE] getproviders.SearchLocalDirectory: .terraform/plugins is a symlink to .terraform/plugins
Initializing provider plugins...
2021/02/25 13:12:56 [TRACE] getproviders.SearchLocalDirectory: found registry.terraform.io/jfrog/artifactory v2.2.6-alpha for darwin_amd64 at .terraform/plugins/registry.terraform.io/jfrog/artifactory/2.2.6-alpha/darwin_amd64
2021/02/25 13:12:56 [TRACE] providercache.fillMetaCache: including .terraform/plugins/registry.terraform.io/jfrog/artifactory/2.2.6-alpha/darwin_amd64 as a candidate package for registry.terraform.io/jfrog/artifactory 2.2.6-alpha
2021/02/25 13:12:56 [DEBUG] Service discovery for registry.terraform.io at https://registry.terraform.io/.well-known/terraform.json
2021/02/25 13:12:56 [TRACE] HTTP client GET request to https://registry.terraform.io/.well-known/terraform.json
- Using previously-installed jfrog/artifactory v2.2.6-alpha
- Finding latest version of hashicorp/xray...
2021/02/25 13:12:56 [DEBUG] GET https://registry.terraform.io/v1/providers/hashicorp/xray/versions
2021/02/25 13:12:56 [TRACE] HTTP client GET request to https://registry.terraform.io/v1/providers/hashicorp/xray/versions
2021/02/25 13:12:57 [TRACE] providercache.fillMetaCache: using cached result from previous scan of .terraform/plugins
2021/02/25 13:12:57 [DEBUG] GET https://registry.terraform.io/v1/providers/-/xray/versions
2021/02/25 13:12:57 [TRACE] HTTP client GET request to https://registry.terraform.io/v1/providers/-/xray/versions
Error: Failed to install provider
Error while installing hashicorp/xray: provider registry registry.terraform.io
does not have a provider named registry.terraform.io/hashicorp/xray
terraform to run:
# Required for Terraform 0.13 and up (https://www.terraform.io/upgrade-guides/0-13.html)
terraform {
required_providers {
artifactory = {
source = "registry.terraform.io/jfrog/artifactory"
version = "2.2.6-alpha"
}
}
}
provider "artifactory" {
url = "http://localhost:8082/artifactory"
username = "admin"
password = "password"
}
resource "xray_policy" "test" {
name = "test-policy-name"
description = "test policy description"
type = "security"
rules {
name = "rule-name"
priority = 1
criteria {
min_severity = "High"
}
actions {
block_download {
unscanned = true
active = true
}
}
}
}
resource "xray_watch" "test" {
name = "test-watch-name"
description = "test watch description"
resources {
type = "all-repos"
name = "All Repositories"
}
assigned_policies {
name = xray_policy.test.name
type = "security"
}
watch_recipients = ["test#example.com"]
}
I figured out it's based on the name of the resource type - so, that it doesn't start with artifactory makes a difference. What I'd like to know is the logic behind this lookup
[1]: https://github.com/ryndaniels/terraform-provider-xray
Terraform v0.13 introduced the idea of third-party providers that belong to other namespaces that are not controlled directly by HashiCorp, but to maximize backward compatibility with modules that were written for Terraform v0.12 and earlier there is a fallback behavior where Terraform will assume that any provider requirement not explicitly declared is aiming to use one of the official providers which now live in the "hashicorp" namespace in the registry, because for Terraform v0.12 and earlier third-party providers were not automatically installable at all.
When writing modules for Terraform v0.13 or later you should include explicit provider requirements to specify the full source addresses for each of the providers your module uses, like this:
terraform {
required_providers {
artifactory = {
source = "jfrog/artifactory"
}
xray = {
source = "ryndaniels/xray"
}
}
}
Unfortunately the "xray" provider you are trying to use here hasn't, at the time of my writing this, been published in the Terraform Registry. The source address ryndaniels/xray I used above is the address this provider would take if the author were to publish it to the registry as-is, because the Terraform Registry uses a methodical naming scheme to generate provider source addresess from GitHub repository addresses.
But unless that provider is later published in the registry, you'll need to install it on your local system manually in order for Terraform to find it and use it. To achieve that, refer to Implied Local Mirror Directories to see which directories Terraform searches for plugins on your platform, and then you can create the necessary directory structure for Terraform to find the provider.
For the sake of an example I'm going to use the $HOME/.terraform.d/plugins/ prefix that Terraform supports on Linux, but note that the paths are different on Windows and macOS and so you will need to adjust to a different path prefix on those operating systems:
$HOME/.terraform.d/plugins/registry.terraform.io/ryndaniels/xray/0.0.1/linux_amd64/terraform-provider-xray
A few more notes about the above:
The registry.terraform.io/ryndaniels/xray part is what Terraform matches with the source you set in your provider_requirements block. registry.terraform.io is the default hostname for a source address that doesn't include a hostname, so ryndaniels/xray is short for registry.terraform.io/ryndaniels/xray.
This particular provider doesn't have any tagged releases and so there isn't a specific version number to use. Because Terraform expects all providers to have a version number, I arbitrarily chose 0.0.1 as the version number in the path above. This will only matter if you write an explicit version argument in your module to constrain which versions are allowed.
I used linux_amd64 as the target platform here, continuing my example of using Linux-style mirror paths. You can find out which platform is correct for you by running terraform version, which on recent Terraform versions includes the name of the platform that your Terraform CLI executable was built for:
Terraform v0.14.4
on linux_amd64
On startup Terraform will scan all of the implied local mirror directories for your current platform and will note any providers it finds there, and assume that you intend to install those from your local filesystem rather than from the registry. Therefore as long as the directory structure is correct as above, terraform init will see that you have ryndaniels/xray installed locally and will use the copy from there rather than asking the Terraform Registry which versions are available for that provider.
If the provider author publishes this provider to the Terraform Registry at a later date, and they do so without renaming the repository or moving it into a different GitHub account, it should appear in the registry as ryndaniels/xray and thus terraform init will be able to automatically install it from there without any further modifications to your modules.
I got suddenly and unexpectedly following error when executing terraform plan.
Error: Error building account: Error getting authenticated object ID: Error parsing json result from the Azure CLI: Error retrieving running Azure CLI: Unable to encode the output with ANSI_X3.4-1968 encoding. U
nsupported characters are discarded.
on main.tf line 4, in provider "azurerm":
4: provider "azurerm" {
Log nearby error looks like this:
2020-04-14T10:22:53.257Z [DEBUG] plugin.terraform-provider-azurerm_v2.5.0_x5: Testing if Service Principal / Client Certificate is applicable for Authentication..
2020-04-14T10:22:53.257Z [DEBUG] plugin.terraform-provider-azurerm_v2.5.0_x5: Testing if Multi Tenant Service Principal / Client Secret is applicable for Authentication..
2020-04-14T10:22:53.257Z [DEBUG] plugin.terraform-provider-azurerm_v2.5.0_x5: Testing if Service Principal / Client Secret is applicable for Authentication..
2020-04-14T10:22:53.257Z [DEBUG] plugin.terraform-provider-azurerm_v2.5.0_x5: Testing if Managed Service Identity is applicable for Authentication..
2020-04-14T10:22:53.257Z [DEBUG] plugin.terraform-provider-azurerm_v2.5.0_x5: Testing if Obtaining a token from the Azure CLI is applicable for Authentication..
2020-04-14T10:22:53.257Z [DEBUG] plugin.terraform-provider-azurerm_v2.5.0_x5: Using Obtaining a token from the Azure CLI for Authentication
2020-04-14T10:22:53.258Z [DEBUG] plugin.terraform-provider-azurerm_v2.5.0_x5: [DEBUG] Resource "https://management.core.windows.net/" isn't for the correct Tenant
2020/04/14 10:22:54 [ERROR] <root>: eval: *terraform.EvalConfigProvider, err: Error building account: Error getting authenticated object ID: Error parsing json result from the Azure CLI: Error retrieving running
Azure CLI: Unable to encode the output with ANSI_X3.4-1968 encoding. Unsupported characters are discarded.
2020/04/14 10:22:54 [ERROR] <root>: eval: *terraform.EvalSequence, err: Error building account: Error getting authenticated object ID: Error parsing json result from the Azure CLI: Error retrieving running Azure
CLI: Unable to encode the output with ANSI_X3.4-1968 encoding. Unsupported characters are discarded.
2020/04/14 10:22:54 [ERROR] <root>: eval: *terraform.EvalOpFilter, err: Error building account: Error getting authenticated object ID: Error parsing json result from the Azure CLI: Error retrieving running Azure
CLI: Unable to encode the output with ANSI_X3.4-1968 encoding. Unsupported characters are discarded.
2020/04/14 10:22:54 [ERROR] <root>: eval: *terraform.EvalSequence, err: Error building account: Error getting authenticated object ID: Error parsing json result from the Azure CLI: Error retrieving running Azure
CLI: Unable to encode the output with ANSI_X3.4-1968 encoding. Unsupported characters are discarded.
2020/04/14 10:22:54 [TRACE] [walkRefresh] Exiting eval tree: provider.azurerm
2020/04/14 10:22:54 [TRACE] vertex "provider.azurerm": visit complete
2020/04/14 10:22:54 [TRACE] dag/walk: upstream of "azurerm_cosmosdb_mongo_database.cupi" errored, so skipping
2020/04/14 10:22:54 [TRACE] dag/walk: upstream of "azurerm_log_analytics_workspace.law-cupi" errored, so skipping
2020/04/14 10:22:54 [TRACE] dag/walk: upstream of "azurerm_cosmosdb_account.cosmodb_account" errored, so skipping
2020/04/14 10:22:54 [TRACE] dag/walk: upstream of "azurerm_cosmosdb_mongo_collection.customer" errored, so skipping
2020/04/14 10:22:54 [TRACE] dag/walk: upstream of "azurerm_resource_group.rg-cupi" errored, so skipping
2020/04/14 10:22:54 [TRACE] dag/walk: upstream of "azurerm_log_analytics_solution.las-cupi" errored, so skipping
2020/04/14 10:22:54 [TRACE] dag/walk: upstream of "azurerm_kubernetes_cluster.aks-cupi" errored, so skipping
2020/04/14 10:22:54 [TRACE] dag/walk: upstream of "azurerm_cosmosdb_mongo_collection.deactivationRequest" errored, so skipping
2020/04/14 10:22:54 [TRACE] dag/walk: upstream of "azurerm_cosmosdb_mongo_collection.customerHash" errored, so skipping
2020/04/14 10:22:54 [TRACE] dag/walk: upstream of "azurerm_cosmosdb_mongo_collection.apiAuth" errored, so skipping
2020/04/14 10:22:54 [TRACE] dag/walk: upstream of "provider.azurerm (close)" errored, so skipping
2020/04/14 10:22:54 [TRACE] dag/walk: upstream of "root" errored, so skipping
and versions of my terraform
$ terraform version
2020/04/14 10:24:24 [INFO] Terraform version: 0.12.24
2020/04/14 10:24:24 [INFO] Go runtime version: go1.12.13
2020/04/14 10:24:24 [INFO] CLI args: []string{"/usr/bin/terraform", "version"}
2020/04/14 10:24:24 [DEBUG] Attempting to open CLI config file: /root/.terraformrc
2020/04/14 10:24:24 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2020/04/14 10:24:24 [INFO] CLI command args: []string{"version"}
Terraform v0.12.24
2020/04/14 10:24:24 [DEBUG] checking for provider in "."
2020/04/14 10:24:24 [DEBUG] checking for provider in "/usr/bin"
2020/04/14 10:24:24 [DEBUG] checking for provider in ".terraform/plugins/linux_amd64"
2020/04/14 10:24:24 [DEBUG] found provider "terraform-provider-azuread_v0.8.0_x4"
2020/04/14 10:24:24 [DEBUG] found provider "terraform-provider-azurerm_v2.5.0_x5"
2020/04/14 10:24:24 [DEBUG] found provider "terraform-provider-random_v2.2.1_x4"
2020/04/14 10:24:24 [DEBUG] found valid plugin: "azurerm", "2.5.0", "/cupi/operations/terraform/.terraform/plugins/linux_amd64/terraform-provider-azurerm_v2.5.0_x5"
2020/04/14 10:24:24 [DEBUG] found valid plugin: "random", "2.2.1", "/cupi/operations/terraform/.terraform/plugins/linux_amd64/terraform-provider-random_v2.2.1_x4"
2020/04/14 10:24:24 [DEBUG] found valid plugin: "azuread", "0.8.0", "/cupi/operations/terraform/.terraform/plugins/linux_amd64/terraform-provider-azuread_v0.8.0_x4"
+ provider.azuread v0.8.0
+ provider.azurerm v2.5.0
+ provider.random v2.2.1
and finally my az cli
$ az --version
azure-cli 2.3.1
command-modules-nspkg 2.0.3
core 2.3.1
nspkg 3.0.4
telemetry 1.0.4
Python location '/opt/az/bin/python3'
Extensions directory '/root/.azure/cliextensions'
Python (Linux) 3.6.5 (default, Apr 1 2020, 07:19:45)
[GCC 7.5.0]
Legal docs and information: aka.ms/AzureCliLegal
My main.tf file:
provider "azuread" {
version = "~>0.8"
}
provider "azurerm" {
version = "~>2"
subscription_id = "..."
features {}
}
terraform {
backend "azurerm" {}
}
I have also read threads bellow. None of which helped or resolved my issue. Same config that doesn't work today, worked with no modification couple of days ago (only thing that could change on client side are versions of plugins - i tried up/down grades but with no success).
https://github.com/terraform-providers/terraform-provider-azurerm/issues/3686
https://github.com/terraform-providers/terraform-provider-azurerm/issues/4906
Terraform with azure CLI - error building account
As mentioned in the comments, The issue was not providing the service principal in the provider.the correct syntax is :
# Configure the Azure Provider
# https://www.terraform.io/docs/providers/azurerm/index.html
provider "azurerm" {
subscription_id = var.SUBSCRIPTION_ID
client_id = var.SP_CLIENT_ID
client_secret = var.SP_CLIENT_SECRET
tenant_id = var.SP_TENANT_ID
version = "=2.0.0" #Can be overide as you wish
features {}
}
What is Service principal?
An Azure service principal is an identity created for use with
applications, hosted services, and automated tools to access Azure
resources. This access is restricted by the roles assigned to the
service principal, giving you control over which resources can be
accessed and at which level. For security reasons, it's always
recommended to use service principals with automated tools rather than
allowing them to log in with a user identity.
More info here.
With that been said, Why we should use service principal with Terraform?
When using Service principal you can give limited permissions to specific resources.
Service Principal is not attached to any user. Therefore, multiple users can use this Service principal.
You can assign permissions to the app identity that is different than your own permissions.
Azure Provider: Authenticating using a Service Principal with a Client Secret.
About AZ CLI login issue:
to be honest I don't have an answer that I feel confident to share. But, my guess is that there is an issue with AZ CLI version 2.3.1.
As you can see about 2 weeks ago when the new version released Azure team fixed an issue related to az login so I guess this is why things are acting differently now.
In case you want to check that, you can downgrade to 2.3.0 and check if this is still happening.
As noted in the official documentation for Terraform on how to authenticate using the Azure CLI, it is recommended to authenticate using personal credentials (through the az cli) when running locally.
We recommend using either a Service Principal or Managed Service Identity when running Terraform non-interactively (such as when running Terraform in a CI server) - and authenticating using the Azure CLI when running Terraform locally.
This becomes a little problematic when you would like to run locally in a Docker container, especially since it seems like the output generated by the az cli have changed its output (intentionally or not), so that Terraform can no longer use it.
As Amit already noted in the accepted answer, this seems to be due to a change, but I would argue that it occurred earlier, since I have to roll all the way back to 2.2.0 (2.2.0-1~bionic on ubuntu) to have it working again.
I had the same issue running terraform in a docker container through an ssh client. I managed to fix it with:
export LC_ALL=en_US.UTF-8
I have created a workspace in Terraform Enterprise by running a terraform init && terraform plan locally with Terraform enterprise set up as my back end:
# Using a single workspace:
terraform {
backend "remote" {
hostname = "dep.app.example.io"
organization = "nnnn"
workspaces {
name = "create-workspace"
}
}
}
Terraform Apply works, and I can launch an ec2 via Terraform Enterprise with this code:
provider "aws" {
region = "${var.region}"
}
resource "aws_instance" "feature" {
count = 1
ami = "${var.ami}"
availability_zone = "${var.availability_zone}"
instance_type = "${var.instance_type}"
tags = {
Name = "${var.name_tag}"
}
}
Now when I run a terraform destroy, I get this error:
Error: error creating run: Invalid Attribute Infrastructure is
not destroyable
The configured "remote" backend encountered an unexpected
error. Sometimes this is caused by network connection problems,
in which case you could retry the command. If the issue
persists please open a support ticket to get help resolving the
problem.
What am I doing wrong here? I want to be able to run a terraform destroy that destroys the infrastructure my new Terraform enterprise workspace spins up.
EDIT: LOGS:
2019/04/03 09:11:54 [INFO] Terraform version: 0.11.11 ac4fff416318bf0915a0ab80e062a99ef3724334
2019/04/03 09:11:54 [INFO] Go runtime version: go1.11.1
2019/04/03 09:11:54 [INFO] CLI args: []string{"/usr/local/bin/terraform", "destroy"}
2019/04/03 09:11:54 [DEBUG] Attempting to open CLI config file: /Users/nlegorrec/.terraformrc
2019/04/03 09:11:54 Loading CLI configuration from /Users/nlegorrec/.terraformrc
2019/04/03 09:11:54 [INFO] CLI command args: []string{"destroy"}
2019/04/03 09:11:54 [TRACE] Preserving existing state lineage "f7abdc54-236c-c906-e701-049f3e2cc00c"
2019/04/03 09:11:54 [TRACE] Preserving existing state lineage "f7abdc54-236c-c906-e701-049f3e2cc00c"
2019/04/03 09:11:54 [DEBUG] Service discovery for dep.app.redbull.com at https://dep.app.redbull.com/.well-known/terraform.json
2019/04/03 09:11:56 [DEBUG] Retrieve version constraints for service tfe.v2 and product terraform
2019/04/03 09:11:57 [INFO] command: backend initialized: *remote.Remote
2019/04/03 09:11:57 [DEBUG] checking for provider in "."
2019/04/03 09:11:57 [DEBUG] checking for provider in "/usr/local/bin"
2019/04/03 09:11:57 [DEBUG] checking for provider in ".terraform/plugins/darwin_amd64"
2019/04/03 09:11:57 [DEBUG] found provider "terraform-provider-aws_v2.4.0_x4"
2019/04/03 09:11:57 [DEBUG] found valid plugin: "aws", "2.4.0", "/Users/nlegorrec/dev/Software Engineering/emp-kpi-tracker_web/dep/.terraform/plugins/darwin_amd64/terraform-provider-aws_v2.4.0_x4"
2019/04/03 09:11:57 [DEBUG] checking for provisioner in "."
2019/04/03 09:11:57 [DEBUG] checking for provisioner in "/usr/local/bin"
2019/04/03 09:11:57 [DEBUG] checking for provisioner in ".terraform/plugins/darwin_amd64"
2019/04/03 09:11:57 [INFO] backend/remote: starting Apply operation
2019/04/03 09:12:00 [DEBUG] plugin: waiting for all plugin processes to complete...
Error: error creating run: Invalid Attribute Infrastructure is not destroyable
The configured "remote" backend encountered an unexpected error. Sometimes
this is caused by network connection problems, in which case you could retry
the command. If the issue persists please open a support ticket to get help
resolving the problem.
Even though its a bit late hopefully this answer can help others in the future.
When using Terraform Enterprise or Terraform Cloud, you need to ensure that you are following their guidance on Destruction and Deletion from within the Workspace
Documentation for this is located here
To queue the destruction of infrastructure that is managed by a workspace you need to ensure that within the Variables of the workspace that you have assigned a variable with the name CONFIRM_DESTROY with a value of 1
Importantly, any changes to the workspace require admin privleges
Once you have completed that you should be able to use the CLI Workflow as you would locally in Terraform.