I've been struggling for a while in Terraform to create an Event Subscription in an Azure Event Grid
As-per screenshot....
EVENT SUBSCRIPTION DETAILS
NAME : EventGrid-Sub1
(don't need to change Event Schema)
TOPIC DETAILS
Event Grid Domain
Topic Resource: EDG-SBX-EventGrid1
Domain Type: EventGrid-DomainTopic1
ENDPOINT DETAILS
Endpoint Type: Event Hubs
Endpoint : eh-sbx-Ingestion
I've been using these as reference, but it seems not only a bit chicken-and-egg, but pieces missing?
https://www.terraform.io/docs/providers/azurerm/r/eventgrid_event_subscription.html
https://www.terraform.io/docs/providers/azurerm/r/eventgrid_topic.html
Has anyone got this working in Terraform?
Thanks in advance
Azure Screenshot on Event Grids / Create Event Subscription screen
#nmca70 There are a couple of ways to achieve this:
Create an ARM template from the final deployment and then run that ARM template using Terraform:
https://www.terraform.io/docs/providers/azurerm/r/template_deployment.html
Create resources in the below order:
Azure event hub: https://www.terraform.io/docs/providers/azurerm/r/eventhub.html
Azure event grid topic: https://www.terraform.io/docs/providers/azurerm/r/eventgrid_topic.html
Azure event grid domain: https://www.terraform.io/docs/providers/azurerm/r/eventgrid_domain.html
Azure event grid subscription: https://www.terraform.io/docs/providers/azurerm/r/eventgrid_event_subscription.html#storage_queue_endpoint
A sample:
resource "azurerm_resource_group" "test" {
name = "resourceGroup1"
location = "West US 2"
}
resource "azurerm_eventhub_namespace" "test" {
name = "acceptanceTestEventHubNamespace"
location = "${azurerm_resource_group.test.location}"
resource_group_name = "${azurerm_resource_group.test.name}"
sku = "Standard"
capacity = 1
kafka_enabled = false
tags = {
environment = "Production"
}
}
resource "azurerm_eventhub" "test" {
name = "acceptanceTestEventHub"
namespace_name = "${azurerm_eventhub_namespace.test.name}"
resource_group_name = "${azurerm_resource_group.test.name}"
partition_count = 2
message_retention = 1
}
resource "azurerm_eventgrid_topic" "test" {
name = "my-eventgrid-topic"
location = "${azurerm_resource_group.test.location}"
resource_group_name = "${azurerm_resource_group.test.name}"
tags = {
environment = "Production"
}
}
resource "azurerm_eventgrid_domain" "test" {
name = "my-eventgrid-domain"
location = "${azurerm_resource_group.test.location}"
resource_group_name = "${azurerm_resource_group.test.name}"
input_schema = "eventgridschema"
input_mapping_fields= {
topic = "my-eventgrid-topic"
}
tags = {
environment = "Production"
}
}
resource "azurerm_eventgrid_event_subscription" "default" {
name = "defaultEventSubscription"
scope = "${azurerm_resource_group.default.id}"
event_delivery_schema = "EventGridSchema"
topic_name = "my-eventgrid-topic"
eventhub_endpoint {
storage_account_id = "${azurerm_eventhub.test.id}"
}
}
Hope this helps!
Related
I'm deploying an AKS cluster via Terraform.
I set an oms_agent block within my aks resource block:
resource "azurerm_kubernetes_cluster" "tfdemo-cluster" {
resource_group_name = var.resourcegroup_name
location = var.location
name = "${var.projectname}-aks"
node_resource_group = "${var.resourcegroup_name}-node"
... omitted to shorten ...
oms_agent {
log_analytics_workspace_id = var.log_analytics_workspace_id
}
Like this it works as aspected.
But when I add an additional resource of type diagnostic_settings like so
resource "azurerm_monitor_diagnostic_setting" "aks-diagnostics" {
name = "aks-logs"
storage_account_id = var.storage_account_id
target_resource_id = azurerm_kubernetes_cluster.tfdemo-cluster.id
log {
category = "kube-audit"
enabled = true
}
metric {
category = "AllMetrics"
retention_policy {
days = 30
enabled = true
}
}
}
I run into an error that says:
"diagnosticsettings.DiagnosticSettingsClient#CreateOrUpdate: Failure sending request: StatusCode=409 -- Original Error: autorest/azure: Service returned an error. Status=nil nil"
When I tried to google that error messages I found issues related to other Azure services where the sku of that service wasn't matching a specified feature or capacity but I'm don't see that here.
Why I want log analytics workspace AND logs dumped into a storage account: My thinking was just that a log anal. ws is really expensive compared to storage in a storage account. So I thought I send say the audit data for long time retention to the cheap storage account (my settings in the given example might not 100% represent that but it's not the point here I'd say) and still have the "expensive" log analytics service to dig into the cluster performance.
Thanks a lot for any input!
I Tried to reproduce the same in my environment to Create an Azure AKS cluster with OMS Agent and Diagnostic Setting using Terraform:
Sending long-term data retention logs to a Azure Storage Account can be more cost-effective than keeping them in a Azure Log Analytics workspace. However, the Azure Log Analytics workspace can still be useful for real-time analysis and performance monitoring.
provider "azurerm" {
features {}
}
resource "azurerm_resource_group" "aksgroup" {
name = "aks-rg"
location = "East US"
}
resource "azurerm_log_analytics_workspace" "oms" {
name = "oms-workspace"
location = azurerm_resource_group.aksgroup.location
resource_group_name = azurerm_resource_group.aksgroup.name
sku = "PerGB2018"
}
resource "azurerm_kubernetes_cluster" "aks" {
name = "cluster-aks1"
location = azurerm_resource_group.aksgroup.location
resource_group_name = azurerm_resource_group.aksgroup.name
dns_prefix = "aks1"
default_node_pool {
name = "default"
node_count = 1
vm_size = "standard_a2_v2"
}
identity {
type = "SystemAssigned"
}
tags = {
Environment = "Production"
}
addon_profile {
oms_agent {
enabled = true
log_analytics_workspace_id = azurerm_log_analytics_workspace.oms.id
}
}
}
output "client_certificate" {
value = azurerm_kubernetes_cluster.aks.kube_config.0.client_certificate
sensitive = true
}
output "kube_config" {
value = azurerm_kubernetes_cluster.aks.kube_config_raw
sensitive = true
}
resource "azurerm_monitor_diagnostic_setting" "aks" {
name = "aks-diagnostic-setting"
target_resource_id = azurerm_kubernetes_cluster.aks.id
storage_account_id = azurerm_storage_account.aks.id
log_analytics_workspace_id = azurerm_log_analytics_workspace.oms.id
log {
category = "kube-audit"
enabled = true
}
metric {
category = "AllMetrics"
retention_policy {
days = 30
enabled = true
}
}
}
resource "azurerm_storage_account" "aks" {
name = "aksdiagnostic"
resource_group_name = azurerm_resource_group.aksgroup.name
location = azurerm_resource_group.aksgroup.location
account_tier = "Standard"
account_replication_type = "LRS"
}
Terraform Apply:
Once ran the code resources are created, like below.
Azure AKS Diagnostic settings created with Log Analytics settings.
Log Analytics settings- created.
I have created an Activity Log Alert in Azure using the following Terraform Code
// We need to define the action group for Security Alerts
resource "azurerm_monitor_action_group" "monitor_action_group_soc" {
name = "sec-alert"
resource_group_name = data.azurerm_resource_group.tenant-global.name
short_name = "sec-alert"
email_receiver {
name = "sendtoAdmin"
email_address = var.email_address
use_common_alert_schema = true
}
}
data "azurerm_monitor_action_group" "monitor_action_group_soc" {
name = "sec-alert"
resource_group_name = var.tenant-global-rg
depends_on = [
azurerm_monitor_action_group.monitor_action_group_soc
]
}
// Monitor Activity Log and Alert
resource "azurerm_monitor_activity_log_alert" "activity_log_alert_cu_security_group" {
name = "Activity Log Alert for Create or Update Security Group"
resource_group_name = data.azurerm_resource_group.ipz12-dat-np-mgmt-rg.name
scopes = [data.azurerm_subscription.current.id]
description = "Monitoring for Create or Update Network Security Group events gives insight into network access changes and may reduce the time it takes to detect suspicious activity"
criteria {
category = "Security"
operation_name = "Microsoft.Network/networkSecurityGroups/write"
}
action {
action_group_id = data.azurerm_monitor_action_group.monitor_action_group_soc.id
}
}
I have created the Network Security Group, added a Rule, deleted the Rule and finally deleted the Network Security Group but I didn't receive any Alerts.
Azure Activity Log Alerts are not working:
These are the modifications I made to your code to achieve the expected result.
provider "azurerm" {
features {}
}
resource "azurerm_resource_group" "<resourcegroup>"{
name = "<resourcegroup>"
location = "Central US"
}
resource "azurerm_monitor_action_group" "<actiongroup>" {
name = "sec-alert"
resource_group_name = "<resourcegroup>"
short_name = "sec-alert"
email_receiver {
name = "xxxxx"
email_address = "xxxxxxx#gmail.com"
use_common_alert_schema = true
}
}
data "azurerm_monitor_action_group" "<actiongroup>" {
name = "sec-alert"
resource_group_name = "<resourcegroup>"
depends_on = [
azurerm_monitor_action_group.<actiongroup>
]
}
resource "azurerm_monitor_activity_log_alert" "azurerm_monitor_activity_log_alert_securitygroup" {
name = "Activity Log Alert for Create or Update Security Group"
resource_group_name = "<resourcegroup>"
scopes = [data.azurerm_subscription.current.id] #My scope is /subscriptions/<subscriptionID>/resourceGroups/<resourcegroup>/providers/Microsoft.Network/networkSecurityGroups/<NetworkSecurityGroup>
description = "Monitoring for Create or Update Network Security Group events gives insight into network access changes and may reduce the time it takes to detect suspicious activity"
criteria {
category = "Security"
operation_name = "Microsoft.Network/networkSecurityGroups/write"
}
action {
action_group_id = data.azurerm_monitor_action_group.<actiongroup>.id
}
}
Created Security alert by running "terraform apply" in AzCLI :
Received a mail once it is added to the Network Security Group:
I have been trying to figure out a way to prepare a terraform template for my app service/az function where I can connect it to application Insight while creating them through Terraform. Well the it worked, BUT the application Insight shows
Migrate this resource to Workspace-based Application Insights to gain support for all of the capabilities of Log Analytics, including Customer-Managed Keys and Commitment Tiers. Click here to learn more and migrate in a few clicks.
How do I acheive it from terraform? As from the documentation page of terraform there is no mention of such setup. Appreciate you help on this.
Here is the terraform code for az-function
resource "azurerm_linux_function_app" "t_funcapp" {
name = "t-function-app"
location = local.resource_location
resource_group_name = local.resource_group_name
service_plan_id = azurerm_service_plan.t_app_service_plan.id
storage_account_name = azurerm_storage_account.t_funcstorage.name
storage_account_access_key = azurerm_storage_account.t_funcstorage.primary_access_key
site_config {
application_stack {
java_version = "11"
}
remote_debugging_enabled = false
ftps_state = "AllAllowed"
}
app_settings = {
APPINSIGHTS_INSTRUMENTATIONKEY = "${azurerm_application_insights.t_appinsights.instrumentation_key}"
}
depends_on = [
azurerm_resource_group.t_rg,
azurerm_service_plan.t_app_service_plan,
azurerm_storage_account.t_funcstorage,
azurerm_application_insights.t_appinsights
]
}
Here is the terraform code for app insight
resource "azurerm_application_insights" "t_appinsights" {
name = "t-appinsights"
location = local.resource_location
resource_group_name = local.resource_group_name
application_type = "web"
depends_on = [
azurerm_log_analytics_workspace.t_workspace
]
}
output "instrumentation_key" {
value = azurerm_application_insights.t_appinsights.instrumentation_key
}
output "app_id" {
value = azurerm_application_insights.t_appinsights.app_id
}
You must create a Log Analytics Workspace and add it to your Application Insights.
For example
resource "azurerm_log_analytics_workspace" "example" {
name = "workspace-test"
location = local.resource_location
resource_group_name = local.resource_group_name
sku = "PerGB2018"
retention_in_days = 30
}
resource "azurerm_application_insights" "t_appinsights" {
name = "t-appinsights"
location = local.resource_location
resource_group_name = local.resource_group_name
workspace_id = azurerm_log_analytics_workspace.example.id
application_type = "web"
}
output "instrumentation_key" {
value = azurerm_application_insights.t_appinsights.instrumentation_key
}
output "app_id" {
value = azurerm_application_insights.t_appinsights.app_id
}
Hope this helps!
I would like to update my exsiting Azure App Service in Terraform by adding a Backup to this App Service.
For now it looks like this:
data "azurerm_app_service_plan" "example" {
name = "MyUniqueServicePlan"
resource_group_name = "example-resources"
}
resource "azurerm_app_service" "example" {
name = "MyUniqueWebAppName"
location = "West Europe"
resource_group_name = "example-resources"
app_service_plan_id = data.azurerm_app_service_plan.example.id
connection_string {
name = "myConectionString"
type = "SQLServer"
value = "Server=tcp:mysqlservername123.database.windows.net,1433;Initial Catalog=MyDatabaseName;Persist Security Info=False;User ID=xxx;Password=xxxxxx;MultipleActiveResultSets=False;Encrypt=True;TrustServerCertificate=False;Connection Timeout=30;"
}
backup {
name = "MyBackupName"
enabled = true
storage_account_url = "https://storageaccountnameqwetih.blob.core.windows.net/mycontainer?sp=r&st=2022-08-31T09:49:17Z&se=2022-08-31T17:49:17Z&spr=https&sv=2021-06-08&sr=c&sig=2JwQ%xx%2B%2xxB5xxxxFZxxVyAadjxxV8%3D"
schedule {
frequency_interval = 30
frequency_unit = "Day"
keep_at_least_one_backup = true
retention_period_in_days = 10
start_time = "2022-08-31T07:11:56.52Z"
}
}
}
But when I run it i got a error A resource with the ID ........ /MyUniqueWebAppName" already exists - to be managed via Terraform this resource needs to be imported into the State.
How in terraform can I point to an existing Azure APP Service and add a backup with the same schedule as I did in my template?
Before you can modify your existing resources with TF, you must import into the terraform state. For this you use import command.
data "azurerm_resource_group" "example" {
name = "<give rg name existing one>"
}
data "azurerm_app_service_plan" "example" {
name = "MyUniqueServicePlan"
resource_group_name = data.azurerm_resource_group.example.name
}
data "azurerm_app_service" "example" {
name = "MyUniqueWebAppName"
location = data.azurerm_resource_group.example.location
resource_group_name = data.azurerm_resource_group.example.name
app_service_plan_id = data.azurerm_app_service_plan.example.id
connection_string {
name = "myConectionString"
type = "SQLServer"
value = "Server=tcp:mysqlservername123.database.windows.net,1433;Initial Catalog=MyDatabaseName;Persist Security Info=False;User ID=xxx;Password=xxxxxx;MultipleActiveResultSets=False;Encrypt=True;TrustServerCertificate=False;Connection Timeout=30;"
}
backup {
name = "MyBackupName"
enabled = true
storage_account_url = "https://storageaccountnameqwetih.blob.core.windows.net/mycontainer?sp=r&st=2022-08-31T09:49:17Z&se=2022-08-31T17:49:17Z&spr=https&sv=2021-06-08&sr=c&sig=2JwQ%xx%2B%2xxB5xxxxFZxxVyAadjxxV8%3D"
schedule {
frequency_interval = 30
frequency_unit = "Day"
keep_at_least_one_backup = true
retention_period_in_days = 10
start_time = "2022-08-31T07:11:56.52Z"
}
}
}
No need to use import command , use this code for your reference
just give rg name existing one in resources group block
i've this terraform code :
# Configure the Azure provider
terraform {
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = "~> 2.65"
}
}
required_version = ">= 0.14.7"
}
provider "azurerm" {
features {}
}
# Generate a random integer to create a globally unique name
resource "random_integer" "ri" {
min = 10000
max = 99999
}
# Create the resource group
resource "azurerm_resource_group" "rg" {
name = "myResourceGroup-${random_integer.ri.result}"
location = "eastus"
}
# Create the Linux App Service Plan
resource "azurerm_app_service_plan" "appserviceplan" {
name = "webapp-asp-${random_integer.ri.result}"
location = azurerm_resource_group.rg.location
resource_group_name = azurerm_resource_group.rg.name
sku {
tier = "Free"
size = "F1"
}
}
# Create the web app, pass in the App Service Plan ID, and deploy code from a public GitHub repo
resource "azurerm_app_service" "webapp" {
name = "webapp-${random_integer.ri.result}"
location = azurerm_resource_group.rg.location
resource_group_name = azurerm_resource_group.rg.name
app_service_plan_id = azurerm_app_service_plan.appserviceplan.id
source_control {
repo_url = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
branch = "master"
manual_integration = true
use_mercurial = false
}
}
This code work as exepected.
now i would try to scale up the SKU of the service plan (to Standard /S1 ), Terraform mark my web app as tainted and say that my web app should be replaced
i try to use the meta argment create_before_destroy in the azure plan definition like this :
resource "azurerm_app_service_plan" "appserviceplan" {
# ...
lifecycle {
create_before_destroy = true
}
}
but it's always ask to recreate the web app
Can someone have a idea about that ?