Im using CIDR filter to check if an IP is inside some subnets.
I need to:
cidr {
id => "netflow-postproc-cidr-src_addr"
address => [ "%{[#metadata][netflow][src_addr]}" ]
network => [ "0.0.0.0/32", "10.0.0.0/8", "172.16.0.0/12", "192.168.0.0/16", "fc00::/7", "127.0.0.0/8", "::1/128", "169.254.0.0/16", "fe80::/10", "224.0.0.0/4", "ff00::/8", "255.255.255.255/32", "::" ]
add_field => { "[#metadata][netflow][src_locality]" => "private" }
}
What I need is to get which subnet has matched IP and not only if "at least one".
For example If i have 172.16.0.10 I need to fill a new field with 172.16.0.0/12
EXAMPLE WHAT I NEED
cidr {
id => "netflow-cidr-src_addr"
address => [ "%{[#metadata][netflow][src_addr]}" ]
network_path => "/etc/logstash/dictionaries/internal_subnet.yml"
add_field => { "[#metadata][netflow][src_subnet]" => "<subnet_that_match>" }
}
If you need each of the 13 networks to be tagged differently then you will need to use 13 different cidr filters.
It seems that there isnt a solution so I have found a workaroud.
Knowing every CIDR, I made a python procedure to create an entry for memcached with every IPs in every subnets.
all_su = Subnet.objects.all()
for iss in all_su:
net = ip_network(iss.cidr)
## ITERATING ALL IP INSIDE CIDR
for i in net:
#creating memcached entry
So in memcahced I have something like this:
subnet-namespace:172.16.0.10-subnet-cidr = 172.16.0.0/12
subnet-namespace:172.16.0.11-subnet-cidr = 172.16.0.0/12
subnet-namespace:172.16.0.12-subnet-cidr = 172.16.0.0/12
[...]
Then in logstash I can check in memcached with default filter:
memcached {
hosts => ["localhost:11211"]
namespace => "subnet-namespace"
get => {
"%{[source][ip]}-subnet-cidr" => "[source][subnet]
}
add_tag => ["source_subnet_cached"]
}
Related
I'm trying to create a merged list containing the output of an aws_subnets maps with appropriate nat gateway ids. The value of a tag is used to determine whether the nat gw id should be merged with the subnet data. I need to do something like the following but the conditional isn't valid. How can I test the values properly?
locals {
nat_gw_subnet_map = {
for nat_gw in aws_nat_gateway.private : [
for subnet in aws_subnet.private : {
if subnet.tags["env"] == nat_gw.tags["env"] :
merge(subnet, {nat_gateway_id = nat_gw.id})
}
]
}
}
Does Terraform have an easy way to get the IP address from the hostname?
Something like this
data "some_data" "fetch_ip" {
url = "https://google.com"
}
resource "null_resource" "temp" {
google_ip = data.some_data.fetch_ip.ip // ipv4: 123.123.123.123
}
Terraform's dns provider provides data sources for reading DNS records for a given host.
If you are looking for the IPv4 addresses for eg google.com then you would need to use the dns_a_record_set data source. In fact, the data source's documentation gives an example that looks up the A record for google.com:
data "dns_a_record_set" "google" {
host = "google.com"
}
output "google_addrs" {
value = "${join(",", data.dns_a_record_set.google.addrs)}"
}
or for HCL2/Terraform 0.12+ syntax:
data "dns_a_record_set" "google" {
host = "google.com"
}
output "google_addrs" {
value = join(",", data.dns_a_record_set.google.addrs)
}
Is it possible to get a list of the public IPs of the NAT gateways in a VPC, using a Terraform data source?
An example of getting a list of subnet ids is shown here, but it is predicated on the aws_subnet_ids data source, which returns a list to start with.
We've got NAT gateways per private subnet. I'm not finding a way to get the list of NAT gateways in a vpc and then get the public IPs from that list.
Has anyone needed and/or solved this issue?
This workaround worked for me
https://github.com/hashicorp/terraform-provider-aws/issues/7575
My code sample
data "aws_nat_gateway" "nat_gw" {
for_each = toset(module.vpc.public_subnets)
subnet_id = each.value
}
Get public IP of NAT to add as source for an Security group
resource "aws_security_group_rule" "allow_https"{
type = "ingress"
security_group_id = module.sg.id
from_port = "443"
to_port = "443"
protocol = "tcp"
cidr_blocks = [ for v in data.aws_nat_gateway.nat_gw : format("${v.public_ip}%s", "/32") ]
}
Try this code.
terraform {
required_providers {
shell = {
source = "scottwinkler/shell"
version = "1.7.10"
}
}
}
provider "shell" {
# Configuration options
}
data "shell_script" "nat_gateways" {
lifecycle_commands {
read = <<-EOF
aws ec2 describe-nat-gateways --region ${var.region}
EOF
}
}
locals {
nat_gw_ips = flatten([
for elem in jsondecode(data.shell_script.nat_gateways.output.NatGateways):
format("${elem.NatGatewayAddresses[0].PublicIp}%s", "/32")
])
}
output "natgwips" {
value = local.nat_gw_ips
}
Following is the terraform script.
variable "vpc_ids" {
default = [
"vpc-**********",
"vpc-**********",
"vpc-**********",
"vpc-**********",
]
type = "list"
}
data "aws_security_groups" "test" {
filter {
name = "vpc-id"
values = "${var.vpc_ids}"
}
}
data "aws_security_group" "selected" {
count = "${length(data.aws_security_groups.test.ids)}"
id = "${element(data.aws_security_groups.test.ids, count.index)}"
}
output "sec_groups" {
value = "${data.aws_security_group.selected.0.description}"
// value = "${join(",", data.aws_security_group.selected.*.description)}"
}
In the last, I am using description but it is not giving the inbound and outbound rules for the security group.
Is anyone know how to ge the inbound and outbound rules using datasource ?
I'm trying to set up a Google Cloud Load Balancer and one step requires updating the named ports on the managed instance groups for which I need a formatted string to generate the command-line call. I feel like this should be something that I can automate with terraform, but I'm struggling with mapping data formats.
I have two sets of source data.
From the instance group resource:
data "google_compute_instance_group" "all" {
count = "${length(var.backend)}"
self_link = "${element(var.backend, count.index)}"
}
I get the existing named ports from data.google_compute_instance_group.all.*.named_port in the format:
[
[
map[name:https port:30443],
map[name:http port:30080]
],
[
map[name:https port:30443],
map[port:30080 name:http]
]
]
I also have the ports that I want to make sure are defined in my own map:
variable "node_ports" {
type = "map"
default = {
"https" = "30443"
"monitor" = "30012"
}
}
There may be overlap between these; I want to select the value defined in the variable. (The named ports are the same for all instance groups.)
First, I want to combine the two maps into a single map to make sure that there is one port for each name. How can I convert the first list of lists of maps to a single map so that I get this?
{
"http" = "30080"
"https" = "30443"
"monitor" = "30012"
}
Second, I want to convert all of that to the format needed on the command line:
gcloud --project ${var.project} compute instance-groups set-named-ports ${basename(var.backend[count.index])} --named-ports=https:443,http:30080,monitor:30012
I think I could do that with a jsonencode hack but I'd be interested in better solutions:
"${replace(jsonencode(named_ports), "/[\\{\\}\"\\s]/", "")}"
I was able to use a for expression to rearrange the list of list of maps using terraform 0.12.x. To break down this statement
merge({ for l in local.lolom[0] : l.name => l.port }, { monitor = local.monitor })
It will go through the first list of maps as denoted by for l in local.lolom[0]
Then it will set the name of the map equal to the port's value returning a map itself so now we have { https = 30443, http = 30080 }
The merge() will then combine our map with the monitor map to give us our intended result
{
"http" = "30080"
"https" = "30443"
"monitor" = "30012"
}
I couldn't find a way to convert the above into the arguments without using jsonencode but the following seems to provide the intended result.
locals {
lolom = [
[
{ name : "https", port : 30443 },
{ name : "http", port : 30080 }
],
[
{ name : "https", port : 30443 },
{ port : 30080, name : "http" }
]
]
monitor = 30012
first = merge({ for l in local.lolom[0] : l.name => l.port }, { monitor = local.monitor })
second = merge({ for l in local.lolom[1] : l.name => l.port }, { monitor = local.monitor })
first_args = replace(jsonencode(local.first), "/[\\{\\}\"\\s]/", "")
second_args = replace(jsonencode(local.second), "/[\\{\\}\"\\s]/", "")
}
output "first_args" {
value = local.first_args
}
output "second_args" {
value = local.second_args
}
Apply
$ terraform apply
Apply complete! Resources: 0 added, 0 changed, 0 destroyed.
Outputs:
first_args = http:30080,https:30443,monitor:30012
second_args = http:30080,https:30443,monitor:30012