In an aks managed slb for standard sku, azure assigns a public ip automatically.
The name of this public ip is auto generated but has the following tags
"tags": {
"aks-managed-type": "aks-slb-managed-outbound-ip"
},
Im unable to retrieve this ip after its created.
The name is also auto generated
"name": "[parameters('publicIPAddresses_837ca1c7_1817_43b7_8f4d_34b750419d4b_name')]",
I tried to filter using the azurerm_public_ip data source and use tags for filtering but this is not working.
data "azurerm_public_ip" "example" {
resource_group_name = "rg-sample-004"
filter {
name = "tag:aks-managed-type"
values = [ "aks-slb-managed-outbound-ip" ]
}
}
This above code is incorrect as the name parameter is not provided, but I don't know the name until its created.
I want to whitelist this IP for the Azure MySQL database i create at apply stage.
Is there any other way to retrieve this public ip during terraform apply?
Here you go, we use this to whitelist access from AKS to key vaults etc:
data "azurerm_public_ip" "aks_outgoing" {
name = join("", (regex("([^/]+)$", join("", azurerm_kubernetes_cluster.aks.network_profile[0].load_balancer_profile[0].effective_outbound_ips))))
resource_group_name = "YOUR_RG"
}
Related
I have written a terraform module for Azure Container Registry (ACR). I would like to have the option to make ACR either publicly available or be available to selected networks only and switch between these two. By selected networks I mean specific subnets or IPs are whitelisted. If no subnet or IP list is provided, the ACR will be public. Otherwise, it will be available via selected networks.
This is how I have defined IP list and subnet list in variables.tf file:
variable "allowed_subnet_ids" {
type = list(string)
description = "List of subnet IDs to be allowed to access the ACR"
}
variable "allowed_ips" {
type = list(string)
description = "White list IP addresses"
}
variable "public_network_access_enabled"{
type = bool
description = "(Optional) Whether public network access is allowed for the container registry. Defaults to true."
}
I have made the network_rule_set property optional by using dynamic block in main.tf as follows:
resource "azurerm_container_registry" "this" {
name = local.acr_name
resource_group_name = var.resource_group_name
location = var.location
sku = var.sku
admin_enabled = var.admin_enabled
public_network_access_enabled = var.public_network_access_enabled
dynamic "network_rule_set" {
for_each = (length(var.allowed_ips) != 0 || length(var.allowed_subnet_ids) != 0) ? [1] : []
content {
default_action = "Deny"
dynamic "virtual_network" {
for_each = var.allowed_subnet_ids
content {
action = "Allow"
subnet_id = virtual_network.value
}
}
dynamic "ip_rule" {
for_each = var.allowed_ips
content {
action = "Allow"
ip_range = ip_rule.value
}
}
}
}
Network_rule_set allows white listing IPs and subnets in ACR and making it optionally public or private by using dynamic block as shown above.
To provide variable values, I have used a terraform.tfvars as follows:
env = "sdbx"
application_id = "appid"
resource_group_name = "rg-sbx"
role = "public"
location = "westeurope"
allowed_ips = [ "84.x.x.x", "51.x.x.x"]
# allowed_ips = []
allowed_subnet_ids = []
public_network_access_enabled = true
Here is the question: There is one serious issue though. If we have a list of IPs to be white listed, it will work. If we later decide to remove IPs from this list or change them, but the list is still not EMPTY, it will work as expected. But if you initiate the ACR with a list of IPs (non-empty list) and later you decide to empty it like
allowed_ips = []
It will skip the block and will not remove those IPs! Does anyone have any solutions for it? I want the block to be able to switch between public and selected networks as shown in Azure portal. In other words I want the dynamic block be able to shrink the IP list to zero when I replace the allowed_ips to an empty list and this way make my ACR public.
Here you can see in the images below how the network should toggle between two states which is my end goal:
ACR available by selected networks and white listed IPs:
ACR available publicly when no allopwed_ips or allowed_subnet_ids are provided should make it a public ACR:
For completeness here is my terraform provider and specifications:
terraform {
required_version = ">= 1.0.0"
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = ">=3.0.0"
}
}
}
provider "azurerm" {
features {}
}
In order to switch between networks when ACR is either publicly available or available to selected networks only, we need to keep a condition for network_rule_set Detailed code as below.
Created a ACR by allowing all Networks
Step1:
Here is the latest code which i have added
**main.tf file as follows: **
provider "azurerm" {
features {}
}
resource "azurerm_container_registry" "acr_name" {
name = "acrswarna"
resource_group_name = var.resource_group_name
location = var.location
sku = var.sku
admin_enabled = var.admin_enabled
// Disable this code block for allowed ip network - Begin
network_rule_set {
default_action = "Allow"
}
// Disable this code block for allowed ip network - End
dynamic "network_rule_set" {
for_each = (length(var.allowed_ips) != 0 || length(var.allowed_subnet_ids) != 0) ? [1] : []
content {
default_action = "Deny"
dynamic "virtual_network" {
for_each = var.allowed_subnet_ids
content {
action = "Allow"
subnet_id = virtual_network.value
}
}
dynamic "ip_rule" {
for_each = var.allowed_ips
content {
action = "Allow"
ip_range = ip_rule.value
}}}}}
variable tf file as
variable "allowed_subnet_ids" {
type = list(string)
description = "List of subnet IDs to be allowed to access the ACR"
}
variable "allowed_ips" {
type = list(string)
description = "White list IP addresses"
}
variable "public_network_access_enabled"{
type = bool
description = "(Optional) Whether public network access is allowed for the container registry. Defaults to true."
}
variable "admin_enabled"{
type = bool
description = "(Optional) Whether public network access is allowed for the container registry. Defaults to true."
}
variable "sku" {
type = string
description = "SKU"
}
variable "resource_group_name" {
type = string
description = "resource_group_name"
}
variable "location" {
type = string
description = "location"
}
Terraform.tfvar file code
env = "sdbx"
application_id = "appid"
resource_group_name = "rg-swarna"
role = "public"
location = "westeurope"
//Disable/ Enable the allowed ips when ever need if it was empty All Network will allow and if any ips its allowed only required range
//allowed_ips = ["84.1.2.3", "51.3.4.2"]
allowed_ips = []
allowed_subnet_ids = []
admin_enabled = true
sku="Premium"
public_network_access_enabled = true
Step2:
Run below commands
terraform plan -var-file .\terraform.tfvars
terraform apply -var-file .\terraform.tfvars -auto-approve
Step3:
After Terraform apply on azure portal we can see the ACR,
Step4:
Updated the code with selected range of IPs and re-run the terraform code
Changes to be done on
Replace below code in terraform.tfvars
//Disable/ Enable the allowed ips when ever need if it was empty All Network will allow and if any ips its allowed only required range
allowed_ips = ["84.1.2.3", "51.3.4.2"]
//allowed_ips = []
main tf file - replace this below code
# // Disable this code block for allowed ip network - Begin
# network_rule_set {
# default_action = "Allow"
# }
# // Disable this code block for allowed ip network - End
Repeat Step2
Here is the output from the portal once after implementation. with selected IP ranges
Roll-back process to keep All Network as default and removed above IP Ranges on portal.
Changes to be done on
Replace below code in terraform.tfvars
//Disable/ Enable the allowed ips when ever need if it was empty All Network will allow and if any ips its allowed only required range
//allowed_ips = ["84.1.2.3", "51.3.4.2"]
allowed_ips = []
main.tf file - replace this below code
// Disable this code block for allowed ip network - Begin
network_rule_set {
default_action = "Allow"
}
// Disable this code block for allowed ip network - End
Repeat Step2
OUTPUT
After terraform apply you can be able to see the traffic route to All Network and removed respective IP address on portal.
I have some terrform code which works, but i want to able to ignore the DNS TXT Record value as this is updated externally using another tool (acme.sh), I have tried multiple differnt types of HCL to ignore the value, the terraform HCL does not fail, just set's the value back to the original value
Any help would be appreciated.
resource "azurerm_resource_group" "mydomain-co-uk-dns" {
name = "mydomain.co.uk-dns"
location = "North Europe"
}
resource "azurerm_dns_zone" "mydomaindns" {
name = "mydomain.co.uk"
resource_group_name = azurerm_resource_group.mydomain-co-uk.name
}
resource "azurerm_dns_txt_record" "_acme-challenge-api" {
name = "_acme-challenge.api"
zone_name = azurerm_dns_zone.mydomaindns.name
resource_group_name = azurerm_resource_group.mydomain-co-uk-dns.name
ttl = 300
record {
value = "randomkey-that-changes externally"
}
tags = {
Environment = "acmesh"
}
lifecycle {
ignore_changes = [
record
]
}
}
Thanks
I tried testing using the same code that you have provided and was successfully able to deploy the resources , then manually changed the value of record for portal and applied the terraform code again and it didn't do any changes just changed the value of the previous record to the newer value changes from portal in the terraform state file.
Note: I used Terraform v1.0.5 on windows_amd64 + provider registry.terraform.io/hashicorp/azurerm v2.83.0.
As confirmed by #Lain , the issue was resolved after upgrading the azurerm from 2.70.0 to latest.
I am using Helm chart provisioned by Terraform which creates Network Load Balancer, but I do not know how to get DNS name of this balancer so I can create Route53 records in Terraform for it.
If I can get it's ARN, I can call it over data block and read dns_name, however there is nothing like thit that Helm can return for me.
Do you have any suggestions?
I would like to keep it as IaC as possible
PS: I am passing some values to Helm chart so it's creating NLB, native functionality of this Chart is to create Classic LB.
service.beta.kubernetes.io/aws-load-balancer-type: nlb
I just found and answer, it's simple using:
Note: I had to specify namespace, otherwise was service null (not found).
data "kubernetes_service" "ingress_nginx" {
metadata {
name = "ingress-nginx-controller"
namespace = "kube-system"
}
}
output "k8s_service_ingress" {
description = "External DN name of load balancer"
value = data.kubernetes_service.ingress_nginx.status.0.load_balancer.0.ingress.0.hostname
}
It can be found in official docs too - https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/data-sources/service
I had to use kubernetes_ingress_v1 so to create a Route 53 entry for the ingress hostname:
data "kubernetes_ingress_v1" "this" {
metadata {
name = "ingress-myservice"
namespace = "myservice"
}
depends_on = [
module.myservice-eks
]
}
resource "aws_route53_record" "this" {
zone_id = local.route53_zone_id
name = "whatever.myservice.com"
type = "CNAME"
ttl = "300"
records = [data.kubernetes_ingress_v1.this.status.0.load_balancer.0.ingress.0.hostname]
}
Using Terraform v0.14.7 I'm creating a cname record set, but when I put the command terraform apply, I got this message error:
Error: update server is not set
In the Azure plataform in the "DNS ZONES" I have already created a zone called: tech.com.br:
provider "azurerm" {
features {}
}
resource "dns_cname_record" "foo" {
zone = "tech.com.br."
name = "foo"
cname = "info.tech.com.br."
ttl = 3600
}
Anyone could help me?
If you have created a DNS zone in the Azure platform, you can manage DNS CNAME Records within Azure DNS by using resource azurerm_dns_cname_record instead of resource "dns_cname_record". Also, we don't need to provide the zone name with the dot . suffix.
Change your code like this:
resource "azurerm_dns_cname_record" "foo" {
zone_name = "tech.com.br"
name = "foo"
record = "info.tech.com.br"
ttl = 3600
resource_group_name = "YourDNSZoneRG"
}
I'm using this repo to create a kubernetes cluster on Azure using acs-engine.
I am wondering if anyone can help me identify how to reference the master VM's public IP address.
This would be used to ssh into the master VM (ssh user#public-ip), which is important because I want to run local-exec provisioners to configure my cluster with Ansible.
I don't believe that it is the first_master_ip in the below main.tf (this is given a value on the repo's variables.tf), though I also don't know how to reference this IP as well.
One other thing that I have tried is to obtain the master VM public IP address using the azure command line, however I also haven't had any success with this because I don't know how to get the cluster-name, which would be passed in with az acs kubernetes browse -g <resource-group-name> -n <cluster-name>
Any help would be greatly greatly appreciated as I've really hit a road block with this.
provider "azurerm" {
subscription_id = "${var.azure_subscription_id}"
client_id = "${var.azure_client_id}"
client_secret = "${var.azure_client_secret}"
tenant_id = "${var.azure_tenant_id}"
}
# Azure Resource Group
resource "azurerm_resource_group" "default" {
name = "${var.resource_group_name}"
location = "${var.azure_location}"
}
resource "azurerm_public_ip" "test" {
name = "acceptanceTestPublicIp1"
location = "${var.azure_location}"
resource_group_name = "${azurerm_resource_group.default.name}"
public_ip_address_allocation = "static"
}
data "template_file" "acs_engine_config" {
template = "${file(var.acs_engine_config_file)}"
vars {
master_vm_count = "${var.master_vm_count}"
dns_prefix = "${var.dns_prefix}"
vm_size = "${var.vm_size}"
first_master_ip = "${var.first_master_ip}"
worker_vm_count = "${var.worker_vm_count}"
admin_user = "${var.admin_user}"
ssh_key = "${var.ssh_key}"
service_principle_client_id = "${var.azure_client_id}"
service_principle_client_secret = "${var.azure_client_secret}"
}
}
# Locally output the rendered ACS Engine Config (after substitution has been performed)
resource "null_resource" "render_acs_engine_config" {
provisioner "local-exec" {
command = "echo '${data.template_file.acs_engine_config.rendered}' > ${var.acs_engine_config_file_rendered}"
}
depends_on = ["data.template_file.acs_engine_config"]
}
# Locally run the ACS Engine to produce the Azure Resource Template for the K8s cluster
resource "null_resource" "run_acs_engine" {
provisioner "local-exec" {
command = "acs-engine generate ${var.acs_engine_config_file_rendered}"
}
depends_on = ["null_resource.render_acs_engine_config"]
}
I have no experience with terraform but acs-engine sets up a lb with a public ip that goes through your master (or balances across multiple masters). You find the ip of that lb by using <dns_prefix>.<region>.cloudapp.azure.com.
But if you need the ip to provision something extra, this won't be enough when you have multiple masters.