I'm creating a an AWS CodeBuild using the following (partial) Terraform Configuration:
resource "aws_codebuild_webhook" "webhook" {
project_name = "${aws_codebuild_project.abc-web-pull-build.name}"
branch_filter = "master"
}
resource "github_repository_webhook" "webhook" {
name = "web"
repository = "${var.github_repo}"
active = true
events = ["pull_request"]
configuration {
url = "${aws_codebuild_webhook.webhook.payload_url}"
content_type = "json"
insecure_ssl = false
secret = "${aws_codebuild_webhook.webhook.secret}"
}
}
for some reason two Webhooks are created on GitHub for that spoken project, one with events pull_request and push, and the second with pull request (the only one I've expected).
I've tried removing the first block (aws_codebuild_webhook) even though terraform documentation give an example with both:
https://www.terraform.io/docs/providers/aws/r/codebuild_webhook.html
but than I'm in a pickle because there isn't a way to acquire the payload_url the Webhook require and currently accept it from aws_codebuild_webhook.webhook.payload_url.
not sure what is the right approach here, Appreciate any suggestion.
Related
I'm trying to setup a Terraform configuration for Sonatype Nexus (among other things). Rather than providing my passwords directly, I want to get them from my 1Password system. The advantage for doing this is this Terraform config will live with alongside my broader infrastructure configuration, which includes the setup of the 1password Connect deployment.
My infrastructure CI/CD therefore already has environment variables set for the 1password credentials out of necessity, and it would be nice to make those the only variables I would need for anything. Hence trying to access this password from 1Password.
Below is my Terraform setup. As you can see, it gets the Nexus admin password from 1Password and tries to use it in the provider. However, when I run this Terraform script, it fails with a 401 response from Nexus when trying to create the blobstore.
To be honest, the 1Password Terraform documentation leaves much to be desired. I don't even know if I can configure a provider with data from another provider to begin with.
terraform {
backend "kubernetes" {
secret_suffix = "nexus-state"
config_path = "~/.kube/config"
}
required_providers {
nexus = {
source = "datadrivers/nexus"
version = "1.21.0"
}
onepassword = {
source = "1Password/onepassword"
version = "1.1.4"
}
}
}
provider "onepassword" {
url = "https://my-1password"
token = var.onepassword_token
}
data "onepassword_item" "nexus_admin" {
vault = "VAULT_UUID"
uuid = "ITEM_UUID"
}
provider "nexus" {
insecure = true
password = data.onepassword_item.nexus_admin.password
username = "admin"
url = "https://my-nexus"
}
resource "nexus_blobstore_file" "npm_private" {
name = "npm-private"
path = "/nexus-data/npm-private"
}
We upgraded to using the integrations/github provider source and ever since we have started getting a 404 when attempting to create a github_repository_webhook with Terraform. I believe we have all of the necessary pieces required based on the docs, but the API uri in the logs is missing the org. NOTE: Real org and repo names have been redacted.
main.tf
resource "aws_codepipeline_webhook" "codepipeline_webhook" {
name = "test-github-webhook"
authentication = "GITHUB_HMAC"
target_action = "CC"
target_pipeline = aws_codepipeline.pipeline.name
authentication_configuration {
secret_token = data.aws_secretsmanager_secret_version.github_token.secret_string
}
filter {
json_path = "$.ref"
match_equals = "refs/heads/{Branch}"
}
tags = merge(var.tags, {
Name = "test-github-webhook"
})
}
# Wire the CodePipeline webhook into a GitHub repository.
resource "github_repository_webhook" "github_webhook" {
repository = "my_repo"
configuration {
url = aws_codepipeline_webhook.codepipeline_webhook.url
content_type = "json"
insecure_ssl = true
secret = data.aws_secretsmanager_secret_version.github_token.secret_string
}
events = ["push"]
}
backend.tf
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "3.65.0"
}
github = {
source = "integrations/github"
version = "~> 4.0"
}
}
}
provider "github" {
token = data.aws_secretsmanager_secret_version.github_token.secret_string
owner = "my_org"
base_url = "https://github.com/my_org/" # we have github enterprise
}
Error on create:
Error: POST https://api.github.com/repos//my_repo/hooks: 404 Not Found []
Note that the org is missing completely from the URL. I've also tried including the org name in the github_repository_webhook resource, but the url still comes out with a double slash and a 404:
Error: POST https://api.github.com/repos//my_org/my_repo/hooks: 404 Not Found []
When I remove the provider source and version completely, terraform falls back to the hashicorp/terraform source and the webhook creates without any issues. Has anyone else run into this problem?
You've probably solved this by now, but just in case there are any others who run into this problem.
This solution assumes you're using Terraform version >=0.13 and the new Terraform GitHub Integration.
Check to make sure that all modules that use the github provider have defined a github required_providers block. If this block is not defined within the module, then it seems that the (deprecated) "hashicorp/github" provider is used even if you've configured this block on the root module where you've defined the github provider. (This is what was causing the error for me.) That is, make sure you've defined the following in each of your modules:
terraform {
required_providers {
github = {
source = "integrations/github"
version = "~> 4.0"
}
}
}
Assuming you've done the above, then try double checking that you've set up the GitHub integration properly. Some notable gotchas are:
When upgrading from hashicorp/github to integrations/github, use terraform state replace-provider. Otherwise, Terraform will still require the old provider to interact with the state file.
The organization argument has been deprecated in favor of the owner argument (should behave the same in the mean time, but just in case).
What is the output of terraform providers?
I suspect you have both versions of the github provider in your state.
If you do,
terraform state replace-provider hashicorp/github integrations/github
Should fix you up.
I ran into this issue a few months ago. Your url is missing the organization. It should be of the format
https://api.github.com/repos/my_organization/my_repo/hooks
What specific syntax needs to be used in the example below in order for Terraform to source the AWS provider from a given path in the local file system instead of requesting a cloud copy from the remote Terraform Registry?
provider "aws" {
region = var._region
access_key = var.access_key
secret_key = var.secret_access_key
}
Something like src=C:\path\to\terraform\aws\provider\binary
I recall Mitchel Hashimoto explaining that this is a new feature during HashiConf, but I cannot seem to find documentation.
You should be able to set it in the CLI configuration as described in the documentation:
provider_installation {
filesystem_mirror {
path = "/usr/share/terraform/providers"
include = ["example.com/*/*"]
}
direct {
exclude = ["example.com/*/*"]
}
}
I'm kind of surprised that I'm running into this problem. I created an aws_wafregional_regex_pattern_set to block incoming requests that contain php in their URI. I expected all requests with php in them to be blocked. However the requests are still making it through. Perhaps, I'm misunderstanding what this resource actually does? I have attached some sample code below.
resource "aws_wafregional_rule" "block_uris_containining_php" {
name = "BlockUrisContainingPhp"
metric_name = "BlockUrisContainingPhp"
predicate {
data_id = "${aws_wafregional_regex_match_set.block_uris_containing_php.id}"
negated = false
type = "RegexMatch"
}
}
resource "aws_wafregional_regex_match_set" "block_uris_containing_php" {
name = "BlockUrisContainingPhp"
regex_match_tuple {
field_to_match {
type = "URI"
}
regex_pattern_set_id = "${aws_wafregional_regex_pattern_set.block_uris_containing_php.id}"
text_transformation = "NONE"
}
}
resource "aws_wafregional_regex_pattern_set" "block_uris_containing_php" {
name = "BlockUrisContainingPhp"
regex_pattern_strings = [ "php$" ]
}
This code creates a String and regex matching condition in AWS WAF. So, I know it's at least getting created. I used cloudwatch to check for blocked requests as I sent requests containing php to the load balancer, but each request went through successfully. Any help with this would be greatly appreciated.
I can't tell from snippet but did you add the rule to web ACL and set the rule action to block?
Also you should try using wafv2 instead of wafregional as wafv2 comes with new features and easier to express rules.
I have App1, App2, App3, etc. To reuse code, I want to create them using Terraform module.
The common infrastructure called by modules is:
root\Common_infra\main.tf:
resource "aws_lambda_function" "app" {
# count = “${var.should_launch}”
function_name = "app"
…
}
resource "aws_cloudwatch_event_rule" "app" {
name = " ${var.app_name } "
schedule_expression = "${var.app_schedule}"
}
resource "aws_cloudwatch_event_target" "app_target" {
rule = "${aws_cloudwatch_event_rule.app.name}"
arn = "${aws_lambda_function.app.arn}"
input = <<EOF
{
"app_name": "${var.app_name}"
}
EOF
}
resource "aws_lambda_permission" "allow_cloudwatch_to_call_lambda" {
action = "lambda:InvokeFunction"
function_name = "${aws_lambda_function.app.function_name}"
principal = "events.amazonaws.com"
source_arn = "${aws_cloudwatch_event_rule.app.arn}"
}
# Other resources each app need to create.
The module for app1 is as follows:
root\app1\main.tf:
module "app1" {
# should_launch = 1
source = "../common_infra"
app_name = "app1"
schedule = "cron(01 01 ? * * *)"
……
}
Using the module, I have successfully launched a cloudwatch event which triggers the lambda at schedule, and I have successfully launched the lambda called “app”. The lambda gets the app_name = app1 as input and then work on app1.
when I create another app2 as follows,
root\app2\main.tf:
module "app2" {
# should_launch = 0
source = "../common_infra"
app_name = "app2"
schedule = "cron(01 01 ? * * *)"
……
}
It tries to create another lambda but fails because the lambda has been created by app1 module. In fact I do not want to create a new lambda because It is unnecessary to create multiple lambda for app1, app2…. I can use input = app_name to control what lambda should do.
I try to use should_launch (see above commented lines) to have lambda created only when app1 is created. But it does not work. When deploying app1, the lambda is created. When creating app2. Terrform complains:
aws_cloudwatch_event_target.app_target
cannot find
arn = "${aws_lambda_function.app.arn}"
My question is: how to organize the layout/structure of my code so that the lambda resource is only declared once for multiple modules? e.g. maybe I should create a new folder root/resource_called_by_all_module/lambda.tf? and then deploy this new folder in advance?
Yeah so as you have pointed out, in the current setup all your modules will try to create all the resources in that file individually. Because the name of your lambda is hard-coded, terraform will rightly complain.
If you have a resource that other things are dependent on like that, you can rearrange your terraform to construct that resource on a separate Terraform apply.
So you could take that Lambda resource out of those files, and place it into a separate folder. Then you simply run Terraform apply for those shared resources first ('app' lambda) before the resources that are dependent on them ('app1','app2').
Once the shared resources are created, you can retrieve the details you need from them by using Terraform Data Sources (usually to get names or ARNs).