I'm wondering if there's an easy way to export all key/pair values in Prefect cloud. Can't find anything in the docs. Thanks!
You could build some bash script to automate that process using those commands:
prefect kv list
prefect kv get foo
Maybe this way?
for kv in `prefect kv list`; do prefect kv get "$kv"; done
Related
Is there an easy way to remove all the previous pipelines runned in Gitlab?
I would like to clean up this section, but didn't find any options through the interface.
Thanks a lot.
You can only delete individual pipelines one by one in the UI.
To do bulk deletions, you can use the pipelines API to programmatically list and delete pipelines.
In Python (with the python-gitlab library) it might look something like this:
import gitlab
project_id = 1234
gl = gitlab.Gitlab('https://gitlab.example.com', private_token='My token')
project = gl.projects.get(project_id)
for pipeline in project.pipelines.list(as_list=False):
pipeline.delete()
I'm discovering Rundeck; is there a way to create and run a Terraform process from inside Rundeck?
Thanks in advance
Regards
You can call your Terraform scripts from the command step or the script step on your workflow in the same way you can call any program. Here you can see a complete example of Rundeck+Terraform integration (and about how to automate your deploys).
UPDATE: Also you can test (and collaborate) with this unofficial plugin.
Using a DevOps pipeline with the Terraform Azure extension from Peter Groenewegen 2.4.0 (Latest).
My question is in regards to setting the TF_LOG=Debug global variable using this extension and troubleshooting in general. I'm getting a vague error message,, "##[error] Terraform failed to execute. Error:" , and wanted to see the debug logs for troubleshooting but haven't been able to do this yet. I've tried using a separate task with an export TF_LOG=Debug or adding it to the global variables section, a tfvars file or right on the terraform apply command(##[command]"terraform" apply -auto-confirm -var 'TF_LOG=DEBUG' -input=false -no-color) with the -var TF_LOG=Debug switch. How can I enable Debug using this extension? Perhaps there is a better way to debug issues like this one? Thanks for any response.
Not sure if this will work for the task you're using, but it's worth a shot:
For the Microsoft terraform task for azure pipelines, this can be achieved by adding the TF_LOG variable to the pipeline with an appropriate value.
If you go your pipeline, then click 'Edit', then go to 'Variables' and add it like so:
I wanted to post this as a feature request but I wanted to see if anyone else has found some clever way of doing this before I post. Or maybe someone from Hashicorp can tell me this will be a feature in the coming
I have looked high and low for some way to mask variables from the console when running terraform apply/show. Preferably trying to mask variables using a local-exec provisioner when passing variables to a script.
A tool called Terrahelp is the only thing I can find that will do this but it will only apply to variables in a tfvars file which doesn't allow interpolations. This doesn't help since we are trying to use Vault to keep secrets out of the terraform files.
Current Versions
Terraform v0.11.7
provider.null v1.0.0
provider.template v1.0.0
provider.vault v1.3.1
provider.vsphere v1.8.1
Use Case
provisioner "local-exec" {
command = "&'${path.module}\\scripts\\script.ps1' -name ${var.node_name} -pass '${var.pass}' -user ${var.user} -server ${var.server}"
interpreter = ["Powershell", "-Command"]
}
Attempted Solutions
I'm using Vault to keep secrets out of the Terraform files, so I am using the vault provider and calling data from it. I have tried to create a module and output the secrets with the sensitive = true value and then calling that module to use the secrets however that still shows in the console.
Proposal
Allow some kind of sensitive value much like output to variables within Terraform. So if scripts like the above are called in the console they won't show sensitive variable information.
References
https://github.com/hashicorp/terraform/issues/16114
https://github.com/hashicorp/terraform/issues/16643
Terraform 13 was released since this question was asked, and allows variables to be marked as sensitive and not shown in the console.
https://www.terraform.io/docs/configuration/outputs.html#sensitive-suppressing-values-in-cli-output
Thanks for the feedback, the passwords cannot be set for a one time use as some of these are service accounts in AD that do other things which those applications cannot handle constant password changes.
We did find a solution through another product which is Azure/Azure DevOps. We are storing credentials in key vaults in Azure which Azure DevOps has access to and using Azure DevOps pipelines to send terraform code to our Build Server. Azure DevOps seems to act as a shell which hides any secrets from the console and it works pretty well. I would recommend it to anyone who is also looking to hide secrets from terraform files/command line.
Here is how I do it for a few on-premise services:
1 - var.password doesn't actually stores a password. Rather, it stores the name of an environment variable.
2 - My scripts get passwords from those environment variables.
3 - I have a small program that loads secrets to environment and clears them for terraform apply.
So in the end I just bypass Terraform for secrets used by scripts. Not ideal, but I also couldn't find a better solution.
I think https://github.com/cloudposse/tfmask might be what you're looking for:
Command line utility to mask sensitive output from a transform plan or terraform apply.
You first set an environment variable to filter the masked keys (admittedly there's some manual work involved here):
export TFMASK_VALUES_REGEX="(?i)^.*(secret|password|oauth|token|key).*$"
And then pipe terraform commands through tfmask, resulting in masked output:
terraform plan | tfmask
This is self-advertising, but I create a tool called terramask that works with Terraform 0.12.
Issues are welcome 🙂
I can't quite tell if this is your use case or not, but one strategy that we've used with sensitive variables is to make use of the default Terraform behavior to use environment variables to set TF variables, e.g.,
variable "sensitive_variable" {
type = "string"
}
sensitive_var=$(curl url/with/remote/value)
export TF_VAR_sensitive_variable=$sensitive_var
terraform apply
I would like to use the jenkins script console some more.
Where do I have to look in order to find a list of available Objects/Methods that I can use via groovy? Is there something online? Should I browse the source on Github? Where would I start?
Like in this example, how would I have known that hudson.model.Hudson.instance.pluginManager.plugins exists and is ready to be called from the jenkins script console?
println(hudson.model.Hudson.instance.pluginManager.plugins)
Thanks!
You are looking for Jenkins Main Module API.
You may find this answer helpful in getting yourself on your way.
You can enter a groovy script in the script console.
The complete API can be found at the jenkins javadoc.