I'm deploying Azure Postgres Flexible Server with Terraform as described in GitHub. The configuration works as expected, no issues there. The only divination from that GitHub template is that I want to configure pgBouncer for Postgres which is now supported natively. I don't see a way how I can create this configuration (i.e., enable this feature).
I've done some research and discovered the configuration feature is not quite available (at least according to the open ticket in GitHub here). At the same time, one of the published replies suggests using azurerm_postgresql_flexible_server_configuration and this resource type is available in Terraform indeed. However, Microsoft documentation states that to enable and configure pgBouncer I need to introduce 7 parameters. I thought that to make the code tidier, I can use a list and foreach loop, like this:
locals {
pg_config = {
"pgbouncer.default_pool_size" : "50",
"pgBouncer.max_client_conn" : "5000",
"pgBouncer.pool_mode" : "TRANSACTION"
etc...
}
}
resource "azurerm_postgresql_flexible_server_configuration" "postgres_fs_config" {
for_each = local.pg_config
name = each.key
value = each.value
server_id = azurerm_postgresql_flexible_server.postgres-fs.id
}
Is this the best way to configure Postgres (without involving CDK)? Did I miss something?
Ok, I verified this approach and it works like a charm. Will stick to it for now.
Related
I'm working on automating the rotation of my azure function app's host key, which is used to maintain a more secure connection between my API Management and my function apps. The issue is that I can not figure out how to accomplish this based on the lack of clear documentation. I found a document for how to create a key for a specific function within the function app, but not for the host level. I've tried using the web ui resource manager to figure out what the proper values are, but host seems to have no values available by GET request to help me see what the formatting needs to be. In fact, I can't find any reference to my function app's host keys anywhere in the resource manager UI. (Of course I can in the portal).
I don't care if it's powershell, bicep, ARM, terraform azapi, whatever, I'd just like to find a way to accomplish the creation of a new hostkey so that I can control it's rotation with terraform. Does anyone know how to accomplish this?
Right now my attempt looks like
resource "azapi_resource" "function_host_key" {
type = "Microsoft.Web/sites/host/functionkeys#2018-11-01"
name = "${azurerm_windows_function_app.api_function.name}-host-key"
parent_id = "${azurerm_windows_function_app.api_function.id}/host"
body = jsonencode({
properties = {
name = "test-key-terraform"
value = "asdfasdfasdfasdfasdfasdfasdf"
}
})
}
I also tried
resource "azapi_resource" "function_host_key" {
type = "Microsoft.Web/sites#2018-11-01"
name = "${azurerm_windows_function_app.api_function.name}-host-key"
parent_id = "${azurerm_windows_function_app.api_function.id}/functionsAppKeys"
location = var.region
}
since it said the body was invalid, but this also throws an error due to there being no body. I'm wondering if this just isn't possible.
I also just tried
resource "azapi_resource" "function_host_key" {
type = "Microsoft.Web/host/functionkeys#2018-11-01"
name = "${azurerm_windows_function_app.api_function.name}-host-key"
parent_id = "${azurerm_windows_function_app.api_function.id}/host"
location = var.region
}
and the result said that it was expecting
parent_id of `parent_id is invalid`: expect ID of `Microsoft.Web/host`
so I'm not sure what that parent_id should be.
I found an example through a bash/powershell script using the azure rest API, but I get a 403 error when I attempt to do it, I can only assume because my function app is secured, but I'm not sure a good way to determine that.
There must be a way to create a key programmatically...
UPDATE
I believe that this has been purposely made impossible now to do with terraform and I need to, as grose and backwards as it may be, use a CLI command in my pipeline. I understand you can do this, but it is (ofc my opinion) that if I am using terraform, I have terraform manage something, not have random CLI commands outside of terraform doing things that TF should be able to manage.
I created a key using az functionapp keys set and that worked, and the output explicitly stated that the type of resource which was created was Microsoft.Web/sites/host/functionKeys, so I went to the Azure Resource Explorer to see what versions were available for this type, since it clearly exists.. and found that nope, azure does not have it listed.
What confuses me is that I see this being done w/ ARM templates and I believe that my code matches theirs, just I'm using AZAPI.. and I get a not found error. Giving up for now
I have 5 different APIs in my Dev environment. This environment was built manually.
However, for the subsequent environments like Test, Pre-Prod, etc.. Terraform is being used.
Since I need to create each of the APIs in the subsequent environments, am extracting each of these APIs as a JSON file, making minor tweaks to the API URLs and importing it into the new environments.
The following is the process that am doing right now.
Went to Resource groups in Azure
Then under API Management service > APIs, clicked on the necessary API
Now, clicked on the three dots next to the API that I need and clicked on Export
Selected OpenAPI v3 (JSON) format
Now, I'm using the extracted JSON file and using the Terraform code below to add it to the APIM
resource "azurerm_api_management_api" "example" {
name = "example-api"
resource_group_name = azurerm_resource_group.example.name
api_management_name = azurerm_api_management.example.name
revision = "1"
display_name = "Example API"
path = "api/path"
protocols = ["https"]
service_url = "https://actualURL-of-the-API"
import {
content_format = "openapi+json"
content_value = file("extracted-filename.json")
}
}
The issue here is:
Even though the API gets added to the APIM, this doesn't create all the data - like Webservice URL, Backend HTTP(s) endpoint
How do I go about doing this?
Are you locked into exporting the Json file and importing it on the other environments through Terraform?
The reason I ask is because I attempted something similar but decided to go another route.
Initially I created the API manually in a Dev environment. I then re-created the same API from the ground up using only Terraform. No Json export & import.
I then used that Terraform script to create my other environments.
That allowed me to bypass the import problem altogether since nothing is imported.
I have found that there are downsides to taking this approach; It is much less intuitive to author the API through the Terraform script than through the Azure GUI. This is therefore more time consuming. Especially since my initial API was discarded for the one generated with the Terraform script.
Additionally, I have had problems with Terraform diffs reporting example changes when there are none (I suspect the same problem is to be had when using the import method).
If you are wondering why I decided to go another route? The reason was twofold; Firstly, similar to you, I had trouble with getting the export/import to generate the API that I wanted. Secondly, I prefer not to rely on auto generated files.
Terraform v0.12.x
I'm trying to create an AWS launch template using of course the aws_launch_template resource, and trying to relate to what the AWS console gives me when I try to create a launch template there. In the AWS console, I see the "Network settings" option seen in screen shot.
However, but I don't see a corresponding setting for the Terraform resource? Is that correct? I think I need to set it because when I try to create a spot fleet request, using Terraform's aws_spot_fleet_request resource, and using the launch template created by Terraform, it defaults to the "EC2-Classic" setting, which doesn't work for me. I get this error
Error: Error requesting spot fleet: InvalidSpotFleetRequestConfig: Invalid: (InstanceType: r5a.xlarge with Os: Linux/UNIX and EC2-Classic)
How can I fix this?
Ah, in the aws_spot_request resource, add an overrides that specifies a subnet id, which will of course put the instances in a VPC
resource "aws_spot_fleet_request" "jenkins_build_fleet" {
...
launch_template_config {
launch_template_specification {
id = module.launch_template.id
version = module.launch_template.version
}
overrides {
subnet_id = "subnet-12345abcde"
}
}
}
According to the documentation, there is already a Gitlab provider in Terraform. However, I can't find a way to create/update secrets in a project. Is that possible? Is there a related upcoming feature?
This would really help us in our work, as many output variables of terraform are required to configure other CI deployments we have.
We are using AWS, and some output variables like IAM credentials or subnet IDs cannot be assumed or deduced easily and at the moment we are struggling to find a way to forward them to our other tools.
Being able to do something like:
provider "gitlab" {
token = "${var.gitlab_token}"
url = "${var.gitlab_url}"
}
# Add a project secret
resource "gitlab_project_secret" "my_project_secret" {
project_name = "my_project"
secret_key = "${env}_AWS_SECRET_ACCESS_KEY"
secret_value = ""${module.my_iam_user_module.secret_access_key}"
}
Would save us a lot of pain.
It can't be done just yet although there is an open issue for it with a comment saying that someone was planning to start working on it a few months ago.
The Gitlab provider uses the upstream xanzy/go-gitlab project which does have support for project variables so it shouldn't be too much work to add it. If you are at all comfortable writing Go and/or have contributed to a Terraform provider before then it might be worth picking it up yourself considering the lack of updates on that issue.
How can I get Terraform 0.10.1 to support two different providers without having to run 'terraform init' every time for each provider?
I am trying to use Terraform to
1) Provision an API server with the 'DigitalOcean' provider
2) Subsequently use the 'Docker' provider to spin up my containers
Any suggestions? Do I need to write an orchestrating script that wraps Terraform?
Terraform's current design struggles with creating "multi-layer" architectures in a single configuration, due to the need to pass dynamic settings from one provider to another:
resource "digitalocean_droplet" "example" {
# (settings for a machine running docker)
}
provider "docker" {
host = "tcp://${digitalocean_droplet.example.ipv4_address_private}:2376/"
}
As you saw in the documentation, passing dynamic values into provider configuration doesn't fully work. It does actually partially work if you use it with care, so one way to get this done is to use a config like the above and then solve the "chicken-and-egg" problem by forcing Terraform to create the droplet first:
$ terraform plan -out=tfplan -target=digitalocean_droplet.example
The above will create a plan that only deals with the droplet and any of its dependencies, ignoring the docker resources. Once the Docker droplet is up and running, you can then re-run Terraform as normal to complete the setup, which should then work as expected because the Droplet's ipv4_address_private attribute will then be known. As long as the droplet is never replaced, Terraform can be used as normal after this.
Using -target is fiddly, and so the current recommendation is to split such systems up into multiple configurations, with one for each conceptual "layer". This does, however, require initializing two separate working directories, which you indicated in your question that you didn't want to do. This -target trick allows you to get it done within a single configuration, at the expense of an unconventional workflow to get it initially bootstrapped.
Maybe you can use a provider instance within your resources/module to set up various resources with various providers.
https://www.terraform.io/docs/configuration/providers.html#multiple-provider-instances
The doc talks about multiple instances of same provider but I believe the same should be doable with distinct providers as well.
A little bit late...
Well, got the same Problem. My workaround is to create modules.
First you need a module for your docker Provider with an ip variable:
# File: ./docker/main.tf
variable "ip" {}
provider "docker" {
host = "tcp://${var.ip}:2375/"
}
resource "docker_container" "www" {
provider = "docker"
name = "www"
}
Next one is to load that modul in your root configuration:
# File: .main.tf
module "docker01" {
source = "./docker"
ip = "192.169.10.12"
}
module "docker02" {
source = "./docker"
ip = "192.169.10.12"
}
True, you will create on every node the same container, but in my case that's what i wanted. I currently haven't found a way to configure the hosts with an individual configuration. Maybe nested modules, but that didn't work in the first tries.