I am quite new to terraform and golang, I am trying to implement a custom provider, for a POC, to check if we can leverage terraform for our own use.
I was able to write and build the golang provider according to this video and some GitHub examples.
I created a go workspace an set the $GOPATH to default, $HOME/go.
Terraform packages is installed at $GOPATH/src/github/hashicorp.
Terraform binary is installed at $HOME/dev and specified at the $PATH.
according to the video, I created the provider package at /terraform/builtin/providers/mycustomprovider
And "go build"ed the package to $GOPATH/bin
Once I tried to 'terraform plan' I get the following:
provider.incapsula: no suitable version installed
version requirements: "(any version)"
versions installed: none
I added the custom provider binary to terraform.d/plugins and tried to run 'terraform plan' again.
now I am getting the following error:
Error: Error asking for user input: 1 error(s) occurred:
* provider.incapsula: fork/exec ~/.terraform.d/plugins/darwin_amd64/terraform-provider-incapsula: permission denied
I tried to chmod to 666 and chown the binary file, but with no luck, I am still getting the same error.
I tried to look for this kind of issue but couldn't find any reference.
I would appreciate any assistance.
Thanks!
The provider binary needs execute permissions, so try using 755 instead of 666. Also if the binary isn't somewhere in your $PATH, you generally need to run `terraform init -plugin-dir=.terraform/plugins/darwin_amd64" so terraform picks up the provider and updates the md5 lock file.
So try chmod 755 <wherever the provider is> and if it's still not working, use terraform init with the -plugin-dir argument pointing to the plugin directory (your provider should already be in there).
Related
I´m trying to setup a pipeline in Azure DevOps which initiates a Azure Resource Group. The configurations for which are saved in a .tf file in the DevOps repository.
The pipeline was created with the classic editor. Following tasks were added to the job agent in the same order: terraform init (Terraform CLI) and Build Project (.NET Core).
Terraform is already installed (file path was added to environment variables).
I´m really new to this and am trying to do my first steps. So any help would be appreciated. Also, you can tell me if any important information is missing.
This is the job agent´s log for the Terraform init task:
2023-01-19T15:15:19.3880770Z ##[section]Starting: terraform init
2023-01-19T15:15:19.3895740Z ==============================================================================
2023-01-19T15:15:19.3896080Z Task : Terraform CLI
2023-01-19T15:15:19.3896240Z Description : Execute terraform cli commands
2023-01-19T15:15:19.3896440Z Version : 0.7.8
2023-01-19T15:15:19.3896590Z Author : Charles Zipp
2023-01-19T15:15:19.3896770Z Help :
2023-01-19T15:15:19.3896890Z ==============================================================================
2023-01-19T15:15:21.3070520Z ##[error]Error: Unable to locate executable file: 'terraform'. Please verify either the file path exists or the file can be found within a directory specified by the PATH environment variable. Also check the file mode to verify the file is executable.
2023-01-19T15:15:21.3186320Z ##[error]Error: Unable to locate executable file: 'terraform'. Please verify either the file path exists or the file can be found within a directory specified by the PATH environment variable. Also check the file mode to verify the file is executable.
2023-01-19T15:15:21.5201550Z ##[section]Finishing: terraform init
I have tried running this in various Backend Type settings. Also have tried to change the Agent specifications multiple times.
Furthermore have I tried runnning this after putting the terraform.exe in the repository root.
My expection was that the pipeline creates a new resource group, but the task won´t even be executed.
It looks like you're using the Azure Pipelines Terraform Tasks extension for Azure DevOps which also includes an installer task for Terraform called "TerraformInstaller". Try adding that to your pipeline before "terraform init" to ensure that Terraform is installed (by default it will install the latest version unless you give it a specific one) and is correct for the agent's OS.
As to why the existing terraform binary isn't being found my guess is that you've been trying to use the windows version of terraform (terraform.exe) instead of the linux version which is just terraform and that the pipeline is running on a linux agent. I'm guessing this because the output of the task says that it can't find the file named terraform without the .exe extension.
Problem solved. Two solutions:
Add Terraform install task into the pipeline before adding other Terraform tasks. You can find useful links for this in the approved answer above.
Run pipeline with Ubuntu (any version I guess) agent (Pipeline -> Agent Specification -> ubuntu-latest).
I prefer the second solution because the ubuntu agent finished the tasks significantly faster than the Microsoft agent (and not because of the added install task which only took 7 seconds).
For comparison:
Ubuntu agent: 1 min 16 sec (first execution; the executions afterwards ~30 sec total)
Windows agent: 4 min 31 sec
I wanted to abandon terraform-bundle binary in exchange for recommended solutions described in here https://github.com/hashicorp/terraform/tree/main/tools/terraform-bundle (we are upgrading terraform to version 1.x atm)
but I’m facing problem with specifying multiple versions of the same provider. In past our code looked like this (when using terraform-bundle):
tf-bundle.hcl:
terraform {
version = "0.14.11"
}
providers {
aws = {
source = "hashicorp/aws"
versions = [""3.75.0", "3.75.2"]
}
and we were able to simply run terraform-bundle and we have two versions of aws provider saved locally. However, it seems like it is not possible with terraform providers mirror command.. How can I store it locally and not to use terrafom-bundle binary?
A single Terraform configuration can only depend on one version of each provider at a time. There is no syntax for specifying multiple versions.
I'm assuming that you're trying to use terraform providers mirror to populate a mirror directory.
That command is designed to work with one configuration at a time and mirror the exact dependencies currently required for that configuration. You can run it multiple times in different working directories with the same target directory to mirror potentially many versions of the same provider, but each single configuration can only possibly contribute one version of each provider.
cd config1
terraform providers mirror ~/.terraform.d/plugins
cd ../config2
terraform providers mirror ~/.terraform.d/plugins
The second call to terraform providers mirror will update ~/.terraform.d/plugins to include additional provider packages if needed, while preserving the ones that were placed there by the first call.
If your goal is only to create a local directory to use as a "filesystem mirror" then you don't need the metadata JSON files that terraform providers mirror generates, since those are only for network mirrors.
In that case, the only requirement is to match one of the two supported directory structures, and so you can potentially construct such a directory without Terraform's help:
mkdir ~/.terraform.d/plugins
cd ~/.terraform.d/plugins
mkdir -p registry.terraform.io/hashicorp/aws/3.75.0/linux_amd64
wget https://releases.hashicorp.com/terraform-provider-aws/3.75.0/terraform-provider-aws_3.75.0_linux_amd64.zip
unzip terraform-provider-aws_3.75.0_linux_amd64.zip -d registry.terraform.io/hashicorp/aws/3.75.0/linux_amd64
wget https://releases.hashicorp.com/terraform-provider-aws/3.75.2/terraform-provider-aws_3.75.2_linux_amd64.zip
unzip terraform-provider-aws_3.75.2_linux_amd64.zip -d registry.terraform.io/hashicorp/aws/3.75.2/linux_amd64
After running the above commands, the ~/.terraform.d/plugins directory will contain two specific versions of the AWS provider. That is one of the implied local mirror directories, so unless you've overridden this behavior with a custom provider_installation block in your CLI configuration Terraform should then install that provider only from this local directory and not from the origin registry.
(I've assumed that you are using Linux above; if not then the details will be slightly different but the effect still the same. Specifically, you'll need to use the correct platform name instead of linux_amd64 in the directory paths and URLs; you can see the platform name for your current platform by running terraform version.)
I'm writing a small blog post regarding TF Provider migration that includes the following commands:
terraform state replace-provider foo/bar foo2/bar2
# Updating TF configuration file
terraform init
Shall I tell users to run
rm -rf .terraform/
rm .terraform.lock.hcl
before running
terraform init
The terraform state replace-provider command requires an initialized backend and so it will not work unless terraform init were run first.
The .terraform directory contains some transient working directory state that Terraform should typically be able to reproduce if needed, but deleting it does mean that any module dependencies that don't have exact version constraints could potentially select a different version when installed again, that command-line-specified backend configuration arguments will be lost, and that the currently-selected workspace will be reset back to the default.
Therefore I would not suggest just casually recommending that users remove that directory, unless you're able to explain the potential consequences of doing so.
.terraform.lock.hcl is the dependency lock file and should be treated as part of the configuration, even though terraform init automatically updates it, since its purpose is to remember between runs which version of each provider was selected. That file should typically be kept under version control once created.
Along with saving a record of which version was selected for each provider, the dependency lock file also provides a "trust on first use" methodology for Terraform providers, by saving the checksums of the providers so that if you take any steps to verify the providers you installed are trustworthy then you can be sure that Terraform will not allow any package with a different checksum unless you specifically opt to upgrade using terraform init -upgrade.
In that case, removing .terraform.lock.hcl would defeat that mechanism by removing the record of the original checksum, meaning there would be no guarantee that terraform init would install the same package as before.
I understand that when I run terraform CLI within a directory, it takes all the terraform artifacts from the current directory.
Is it possible to have terraform apply and terraform plan to look for scripts in a directory other than current PWD such that I don't have to keep changing current directory?
At the time I'm writing this, Terraform v0.14.0 is currently in release candidate and its final release is expected in the next few weeks.
That new version will introduce a new feature to allow requesting that Terraform switch directory itself before running any of its subcommands. For example:
terraform -chdir=subdirectory init
terraform -chdir=subdirectory apply
This is essentially the same as launching Terraform with the current working directory already set to the given subdirectory, but with two minor differences:
Terraform will set the path.cwd value to be the directory where Terraform was run from, rather than the subdirectory. As before, path.root is the directory containing the root module, so both directories are available.
If your CLI configuration includes paths that Terraform resolves during its startup then they will be resolved relative to the original working directory, because the directory switch applies only to the behavior of the given subcommand, not to actions Terraform takes prior to running the subcommand.
Neither of these differences are usually significant for common Terraform use, so in most cases you can think of the -chdir= option as having the same effect as switching into the target directlry using cd first.
I am having a problem with terraform init, and terrafor get. I wanted terraform to download my modules freshly so I removed .terraform folder. But when I tried terraform init, it always gave me something like:
Error downloading modules: Error loading modules: error downloading 'ssh://git#mygitaddr': C:\cygwin64\bin\git.exe exited with 128: fatal: could not create work tree dir '.terraform\modules\b58395849f04859395839': No such file or directory.
It seems like terraform is still looking for my old module. I have spent some time searching online and suspect that this is somehow related to symlink but I'm not sure. However, it had no problem running in intellij with same setup using Run/Debug configurations. It also worked in windows cmd prompt.
I'm using terraform 0.11
please try the following steps.
1.) start Cygwin terminal and go to your git/terraform project:
$ cd git/<PROJECT>/<FOLDER with FILES.tf>
2.) now create terraform-folder and subfolder:
$ mkdir -p .terraform/modules/
3.) execute:
$ terraform get