I want to get a list of all the variables (optional or not) and ideally their defaults defined in a resource. Say in the AWS provider, but need anything. I know I can get and look at the documentation (which doesn't list everything I want) or the raw provider code and find all this, but is there a utility that would list this for me?
The schemas-extractor from the Hashicorp Terraform / HCL Language plugin source builds a machine readable schema document for all providers in the terraform-providers org on GitHub.
The Terraform LSP is another likely source of this information.
Related
I have a couple apps that use the same GCP project. There are dev, stage, and prod projects, but they're basically the same, apart from project IDs and project numbers. I would like to have a repo in gitlab like config where I keep these IDs, in a dev.tfvars, stage.tfvars, prod.tfvars. Currently each app's repo has a directory of config/{env}.tfvars, which is really repetitive.
Googling for importing or including terraform resources is just getting me results about terraform state, so hasn't been fruitful.
I've considered:
Using a group-level Gitlab variable just as key=val env file and have my gitlab-ci yaml source the correct environment's file, then just include what I need using -var="key=value" in my plan and apply commands.
Creating a terraform module that either uses TF_WORKSPACE or an input prop to return the correct variables. I think this may be possible, but I'm new to TF, so I'm not sure how to return data back from a module, or if this type of "side-effects only" solution is an abusive workaround to something there's a better way to achieve.
Is there any way to include terraform variables from another Gitlab project?
I am trying to run this code:
locals {
terraform_modules_git = "git::ssh://....#vs-ssh.visualstudio.com/v3/...../terraform-modules"
terraform_modules_module = "resource_group?ref=v15.0.0"
}
module "MyModuleCall" {
source = "${local.terraform_modules_git}/${local.terraform_modules_module}"
}
My goal was to consolidate all Tag references in one place and not duplicate long string with the name of the repo with all my modules numerous times.
And I get this error:
Error: Variables not allowed
on main.tf line 12, in module "MyModuleCall":
12: source = "${local.terraform_modules_git}/${local.terraform_modules_module}"
Variables may not be used here.
Does anybody know why they have put this limitation? What is wrong with using variables?
Does anybody see any work around?
You can't dynamically generate source. You must explicitly hardcode it, as explained in the docs:
This value must be a literal string with no template sequences; arbitrary expressions are not allowed.
Sadly, I'm not aware of any workaround, except pre-processing templates before using them. The pre-processing would just find and replace the source with what you want.
Dependencies in Terraform are handled statically before executing the program, because the runtime needs to have access to all of the involved code (in Terraform's case, both modules and providers) before it can create a runtime context in order to execute any code. This is similar to most other programming languages, where you'd typically install dependencies using a separate command like pip install or npm install or go get before you can run the main program. In Terraform's case, the dependency installer is terraform init, and "running the program" means running terraform plan or terraform apply.
For this reason, Terraform cannot and does not permit dynamically-constructed module or provider source addresses. If you need to abstract the physical location and access method of your modules from the address specified in calling modules then one option is to use a module registry to tell Terraform how to map a local address like yourcompany.example.com/yourteam/resource-group/azure to the more complex Git URL that Terraform will actually use to fetch it.
However, in practice most teams prefer to specify their Git URLs directly because it results in a simpler overall system, albeit at the expense of it being harder to move your modules to a new location at a later date. A compromise between these two is to use a hosted service which provides Terraform Registry services, such as Terraform Cloud, but of course that comes at the expense of introducing another possible point of failure into your overall system.
I wanted to post this as a feature request but I wanted to see if anyone else has found some clever way of doing this before I post. Or maybe someone from Hashicorp can tell me this will be a feature in the coming
I have looked high and low for some way to mask variables from the console when running terraform apply/show. Preferably trying to mask variables using a local-exec provisioner when passing variables to a script.
A tool called Terrahelp is the only thing I can find that will do this but it will only apply to variables in a tfvars file which doesn't allow interpolations. This doesn't help since we are trying to use Vault to keep secrets out of the terraform files.
Current Versions
Terraform v0.11.7
provider.null v1.0.0
provider.template v1.0.0
provider.vault v1.3.1
provider.vsphere v1.8.1
Use Case
provisioner "local-exec" {
command = "&'${path.module}\\scripts\\script.ps1' -name ${var.node_name} -pass '${var.pass}' -user ${var.user} -server ${var.server}"
interpreter = ["Powershell", "-Command"]
}
Attempted Solutions
I'm using Vault to keep secrets out of the Terraform files, so I am using the vault provider and calling data from it. I have tried to create a module and output the secrets with the sensitive = true value and then calling that module to use the secrets however that still shows in the console.
Proposal
Allow some kind of sensitive value much like output to variables within Terraform. So if scripts like the above are called in the console they won't show sensitive variable information.
References
https://github.com/hashicorp/terraform/issues/16114
https://github.com/hashicorp/terraform/issues/16643
Terraform 13 was released since this question was asked, and allows variables to be marked as sensitive and not shown in the console.
https://www.terraform.io/docs/configuration/outputs.html#sensitive-suppressing-values-in-cli-output
Thanks for the feedback, the passwords cannot be set for a one time use as some of these are service accounts in AD that do other things which those applications cannot handle constant password changes.
We did find a solution through another product which is Azure/Azure DevOps. We are storing credentials in key vaults in Azure which Azure DevOps has access to and using Azure DevOps pipelines to send terraform code to our Build Server. Azure DevOps seems to act as a shell which hides any secrets from the console and it works pretty well. I would recommend it to anyone who is also looking to hide secrets from terraform files/command line.
Here is how I do it for a few on-premise services:
1 - var.password doesn't actually stores a password. Rather, it stores the name of an environment variable.
2 - My scripts get passwords from those environment variables.
3 - I have a small program that loads secrets to environment and clears them for terraform apply.
So in the end I just bypass Terraform for secrets used by scripts. Not ideal, but I also couldn't find a better solution.
I think https://github.com/cloudposse/tfmask might be what you're looking for:
Command line utility to mask sensitive output from a transform plan or terraform apply.
You first set an environment variable to filter the masked keys (admittedly there's some manual work involved here):
export TFMASK_VALUES_REGEX="(?i)^.*(secret|password|oauth|token|key).*$"
And then pipe terraform commands through tfmask, resulting in masked output:
terraform plan | tfmask
This is self-advertising, but I create a tool called terramask that works with Terraform 0.12.
Issues are welcome 🙂
I can't quite tell if this is your use case or not, but one strategy that we've used with sensitive variables is to make use of the default Terraform behavior to use environment variables to set TF variables, e.g.,
variable "sensitive_variable" {
type = "string"
}
sensitive_var=$(curl url/with/remote/value)
export TF_VAR_sensitive_variable=$sensitive_var
terraform apply
For what I can understand one can build Logic Apps with Terraform. However, the docs are still not very good, and it looks like this feature is pretty new.
What are the limitations when it comes to TF and Azure Logic Apps? Are there any?
I want to build a two apps, one that is triggered every month and another that is triggered by a https request. I want these then to run two python scripts, and I want the later one to return the result from this script to the client that called the https.
Is this possible to automate in Terraform? At this moment, there are very little examples and documentation on this. Any comment or tip is helpful and greeted with open arms!
You can create a blank Logic App instance through Terrform (TF). But, if you want to add triggers and actions, I wouldn't recommend using TF at all, as of the provider version of 1.20.0.
TF lacks document around parameters. As you know there are two parameters properties – right under the properties property and right under the definitions property. This document states parameters but it doesn't clearly say which one. I'm guessing this refers to the one under the definitions property, but it actually doesn't work – throws Invalid Template error without enough explanation.
UPDATE: I just reverse engineered by importing a Logic App instance using terraform import. The parameters is actually pointing to the one under the properties property. However, it still doesn't work as the Logic App's parameter value can be anything – object, string, integer, etc, while TF's parameter expects string only. Also, there is no way to create parameters under the definitions property.
TF only supports two triggers – HTTP trigger and Timer trigger. All other triggers should use the azurerm_logic_app_trigger_custom resource, but it requires the body part to manually write a JSON object or import from a file, which can't be parameterised through variables or locals.
TF only supports one action – HTTP action. All other actions should use the azurerm_logic_app_action_custom resource, but, like the same issue above, it's not that useful.
In conclusion, TF lacks supports parameters, triggers and actions. So, unless you just create a blank Logic App instance, TF wouldn't be an option for Logic Apps. If you still want to create a blank Logic App instance with TF, then I would recommend this approach using Azure PowerShell or Azure CLI.
For clarity, you don't use Terraform to create LogicApps. LogicApps are designed in either the Portal or Visual Studio.
Terraform is a deployment/management tool. You can almost surely deploy your LogicApps, and other resources, with Terraform, but they're are already created.
isn't the point of terraform to stand up resources across various environments just by passing in -var environment=qa to create a qa instance of the logic app? prod? uat? marcplaypen? I was hoping to use terraform import to create the terraform file, then create multiple versions of it. I can do it, but not with any parameters, which breaks one of my 'actions'.
I was using a combo of terraform import and logic app code view. most of my actions are pretty much a combo of copying the json block for each action, and modifying based on the first entry of the 'body' of the action generated from terraform import.
Then setting up the dependencies manually based off of runAfter, which tells me what an action is dependent on.
but, it fails on parameters, complaining there's only these declared parameters for my definition are ''.'
Is there a way to use an enterprise Github repo with sub-directories as a Terraform module source? Reading the Module source doc, I'm attempting to use the Generic Git repository type, however, it does not appear as though its possible to access sub-directories using this method. E.g.,
module "awesome" {
source = "git::https://github.enterprise.com/org/repo.git//path//to//module"
# Note: I've also tried without the "forced source type."
}
Unfortunately terraform get downloads the entire repository contents.
I can't say that I would expect this to work (given that its not explicitly supported) but perhaps someone out there has figured it out. If not, perhaps it could be a feature request.