FluentBit setup - terraform

I'm trying to set up FluentBit for my EKS cluster in Terraform, via this module, and I have couple of questions:
cluster_identity_oidc_issuer - what is this? Frankly, I was just told to set this up, so I have very little knowledge about FluentBit, but I assume this "issuer" provides an identity with needed permissions. For example, Okta? We use Okta, so what would I use as a value in here?
cluster_identity_oidc_issuer_arn - no idea what this value is supposed to be.
worker_iam_role_name - as in the role with autoscaling capabilities (oidc)?
This is what eks.tf looks like:
module "eks" {
source = "terraform-aws-modules/eks/aws"
cluster_name = "DevOpsLabs"
cluster_version = "1.19"
cluster_endpoint_private_access = true
cluster_endpoint_public_access = true
cluster_addons = {
coredns = {
resolve_conflicts = "OVERWRITE"
}
kube-proxy = {}
vpc-cni = {
resolve_conflicts = "OVERWRITE"
}
}
vpc_id = "xxx"
subnet_ids = ["xxx","xxx", "xxx", "xxx" ]
self_managed_node_groups = {
bottlerocket = {
name = "bottlerocket-self-mng"
platform = "bottlerocket"
ami_id = "xxx"
instance_type = "t2.small"
desired_size = 2
iam_role_additional_policies = ["arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore"]
pre_bootstrap_user_data = <<-EOT
echo "foo"
export FOO=bar
EOT
bootstrap_extra_args = "--kubelet-extra-args '--node-labels=node.kubernetes.io/lifecycle=spot'"
post_bootstrap_user_data = <<-EOT
cd /tmp
sudo yum install -y https://s3.amazonaws.com/ec2-downloads-windows/SSMAgent/latest/linux_amd64/amazon-ssm-agent.rpm
sudo systemctl enable amazon-ssm-agent
sudo systemctl start amazon-ssm-agent
EOT
}
}
}
And for the role.tf:
data "aws_iam_policy_document" "cluster_autoscaler" {
statement {
effect = "Allow"
actions = [
"autoscaling:DescribeAutoScalingGroups",
"autoscaling:DescribeAutoScalingInstances",
"autoscaling:DescribeLaunchConfigurations",
"autoscaling:DescribeTags",
"autoscaling:SetDesiredCapacity",
"autoscaling:TerminateInstanceInAutoScalingGroup",
"ec2:DescribeLaunchTemplateVersions",
]
resources = ["*"]
}
}
module "config" {
source = "github.com/ahmad-hamade/terraform-eks-config/modules/eks-iam-role-with-oidc"
cluster_name = module.eks.cluster_id
role_name = "cluster-autoscaler"
service_accounts = ["kube-system/cluster-autoscaler"]
policies = [data.aws_iam_policy_document.cluster_autoscaler.json]
tags = {
Terraform = "true"
Environment = "dev-test"
}
}

Since you are using a Terraform EKS module, you can access attributes of the created resources by looking at the Outputs tab [1]. There you can find the following outputs:
cluster_id
cluster_oidc_issuer_url
oidc_provider_arn
They are accessible by using the following syntax:
module.<module_name>.<output_id>
In your case, you would get the values you need using the following syntax:
cluster_id -> module.eks.cluster_id
cluster_oidc_issuer_url -> module.eks.cluster_oidc_issuer_url
oidc_provider_arn -> module.eks.oidc_provider_arn
and assign them to the inputs from the FluentBit module:
cluster_name = module.eks.cluster_id
cluster_identity_oidc_issuer = module.eks.cluster_oidc_issuer_url
cluster_identity_oidc_issuer_arn = module.eks.oidc_provider_arn
For the worker role I didn't see an output from the eks module, so I think that could be an output of the config module [2]:
worker_iam_role_name = module.config.iam_role_name
The OIDC parts of configuration are coming from the EKS cluster [3]. Another blog post going in details can be found here [4].
[1] https://registry.terraform.io/modules/terraform-aws-modules/eks/aws/latest?tab=outputs
[2] https://github.com/ahmad-hamade/terraform-eks-config/blob/master/modules/eks-iam-role-with-oidc/outputs.tf
[3] https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html
[4] https://aws.amazon.com/blogs/containers/introducing-oidc-identity-provider-authentication-amazon-eks/

Related

What to do if instance creating and cloud-config in separate session in terraform?

I was able to manually create this device instance in OpenStack, and now I am trying to see how to make it work by terraform.
Anyway, this device instance need to do a hard reboot after the volume attachment, and Any cloud-config needs be down after rebooting. Here is the general sketch of my current main.tf file.
# Configure the OpenStack Provider
terraform {
required_providers {
openstack = {
source = "terraform-provider-openstack/openstack"
}
}
}
data "template_file" "user_data"{
template = file("./userdata.yaml")
}
# create an instance
resource "openstack_compute_instance_v2" "server" {
name = "Device_Instance"
image_id = "xxx..."
image_name = "device_vmdk_1"
flavor_name = "m1.tiny"
key_pair = "my-keypair"
region = "RegionOne"
config_drive = true
network {
name = "main_network"
}
}
resource "openstack_blockstorage_volume_v2" "volume_2" {
name = "device_vmdk_2"
size = 1
image_id = "xxx...."
}
resource "openstack_blockstorage_volume_v3" "volume_3" {
name = "device_vmdk_3"
size = 1
image_id = "xxx..."
}
resource "openstack_compute_volume_attach_v2" "va_1" {
instance_id = "${openstack_compute_instance_v2.server.id}"
volume_id = "${openstack_blockstorage_volume_v2.volume_2.id}"
}
resource "openstack_compute_volume_attach_v2" "va_2" {
instance_id = "${openstack_compute_instance_v2.server.id}"
volume_id = "${openstack_blockstorage_volume_v3.volume_3.id}"
}
resource "null_resource" "reboot_instance" {
provisioner "local-exec" {
on_failure = fail
interpreter = ["/bin/bash", "-c"]
command = <<EOT
openstack server reboot --hard Device_Instance --insecure
echo "................"
EOT
}
depends_on = [openstack_compute_volume_attach_v2.va_1, openstack_compute_volume_attach_v2.va_2]
}
resource "openstack_compute_instance_v2" "server_config" {
name = "Device_Instance"
user_data = data.template_file.user_data.rendered
depends_on = [null_resource.reboot_instance]
}
As of now, it was able to:
have the "Device-Cloud-Instance" generated.
have the "Device-Cloud-Instance" hard-rebooted.
But it fails after rooting. As you may find, I have added this section in the end, but it seems not working.
resource "openstack_compute_instance_v2" "server_config"{}
Any ideas how to make it work?
Thanks,
Jack

Retrieve IP address from instances using for_each

I have this script which works great. It created 3 instances with the sepcified tags to identify them easily. But issue is i want to add a remote-exec provisioner (currently commented) to the code to install some packages. If i was using count, i could have looped over it to do the remote-exec over all the instances. I could not use count because i had to use for_each to loop over a local list. Since count and for_each cannot be used together, how do i loop over the instances to retrieve their IP addresses for using in the remote-exec provisioner.
On digital ocean and AWS, i was able to get it work using host = "${self.public_ip}"
But it does not work on vultr and gives the Unsupported attribute error
instance.tf
resource "vultr_ssh_key" "kubernetes" {
name = "kubernetes"
ssh_key = file("kubernetes.pub")
}
resource "vultr_instance" "kubernetes_instance" {
for_each = toset(local.expanded_names)
plan = "vc2-1c-2gb"
region = "sgp"
os_id = "387"
label = each.value
tag = each.value
hostname = each.value
enable_ipv6 = true
backups = "disabled"
ddos_protection = false
activation_email = false
ssh_key_ids = [vultr_ssh_key.kubernetes.id]
/* connection {
type = "ssh"
user = "root"
private_key = file("kubernetes")
timeout = "2m"
host = vultr_instance.kubernetes_instance[each.key].ipv4_address
}
provisioner "remote-exec" {
inline = "sudo hostnamectl set-hostname ${each.value}"
} */
}
locals {
expanded_names = flatten([
for name, count in var.host_name : [
for i in range(count) : format("%s-%02d", name, i + 1)
]
])
}
provider.tf
terraform {
required_providers {
vultr = {
source = "vultr/vultr"
version = "2.3.1"
}
}
}
provider "vultr" {
api_key = "***************************"
rate_limit = 700
retry_limit = 3
}
variables.tf
variable "host_name" {
type = map(number)
default = {
"Manager" = 1
"Worker" = 2
}
}
The property you are looking for is called main_ip instead of ip4_address or something like that. Specifically accessible via self.main_ip in your connection block.

Terraform API Gateway HTTP API - Getting the error Insufficient permissions to enable logging

My terraform script for deploying an HTTP API looks like the following. I am getting the following error when I run this -
error creating API Gateway v2 stage: BadRequestException: Insufficient permissions to enable logging
Do I need to add something else to make it work?
resource "aws_cloudwatch_log_group" "api_gateway_log_group" {
name = "/aws/apigateway/${var.location}-${var.custom_tags.Layer}-demo-publish-api"
retention_in_days = 7
tags = var.custom_tags
}
resource "aws_apigatewayv2_api" "demo_publish_api" {
name = "${var.location}-${var.custom_tags.Layer}-demo-publish-api"
description = "API to publish event payloads"
protocol_type = "HTTP"
tags = var.custom_tags
}
resource "aws_apigatewayv2_vpc_link" "demo_vpc_link" {
name = "${var.location}-${var.custom_tags.Layer}-demo-vpc-link"
security_group_ids = local.security_group_id_list
subnet_ids = local.subnet_ids_list
tags = var.custom_tags
}
resource "aws_apigatewayv2_integration" "demo_apigateway_integration" {
api_id = aws_apigatewayv2_api.demo_publish_api.id
integration_type = "HTTP_PROXY"
connection_type = "VPC_LINK"
integration_uri = var.alb_listener_arn
connection_id = aws_apigatewayv2_vpc_link.demo_vpc_link.id
integration_method = "POST"
timeout_milliseconds = var.api_timeout_milliseconds
}
resource "aws_apigatewayv2_route" "demo_publish_api_route" {
api_id = aws_apigatewayv2_api.demo_publish_api.id
route_key = "POST /api/event"
target = "integrations/${aws_apigatewayv2_integration.demo_apigateway_integration.id}"
}
resource "aws_apigatewayv2_stage" "demo_publish_api_default_stage" {
depends_on = [aws_cloudwatch_log_group.api_gateway_log_group]
api_id = aws_apigatewayv2_api.demo_publish_api.id
name = "$default"
auto_deploy = true
tags = var.custom_tags
route_settings {
route_key = aws_apigatewayv2_route.demo_publish_api_route.route_key
throttling_burst_limit = var.throttling_burst_limit
throttling_rate_limit = var.throttling_rate_limit
}
default_route_settings {
detailed_metrics_enabled = true
logging_level = "INFO"
}
access_log_settings {
destination_arn = aws_cloudwatch_log_group.api_gateway_log_group.arn
format = jsonencode({ "requestId":"$context.requestId", "ip": "$context.identity.sourceIp"})
}
}
I was stuck on this for a couple of days before reaching out to AWS support. If you have been deploying a lot of HTTP APIs, then you might have run into the same issue where an IAM policy gets very large.
Run this AWS CLI command to find the associated CloudWatch Logs resource policy:
aws logs describe-resource-policies
Look for AWSLogDeliveryWrite20150319. You'll notice this policy has a large number of associated LogGroup resources. You have three options:
Adjust this policy by removing some of the potentially unused entries.
Change the resource list to "*"
You can add another policy. Based on this policy, split the resource records between them.
Apply updates via this AWS CLI command:
aws logs put-resource-policy
Here's the command I ran to set resources. Use "*" for the policy:
aws logs put-resource-policy --policy-name AWSLogDeliveryWrite20150319 --policy-document "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Sid\":\"AWSLogDeliveryWrite\",\"Effect\":\"Allow\",\"Principal\":{\"Service\":\"delivery.logs.amazonaws.com\"},\"Action\":[\"logs:CreateLogStream\",\"logs:PutLogEvents\"],\"Resource\":[\"*\"]}]}"
#Marcin Your initial comment about the aws_api_gateway_account was correct. I added the following resources and now it is working fine -
resource "aws_api_gateway_account" "demo" {
cloudwatch_role_arn = var.apigw_cloudwatch_role_arn
}
data "aws_iam_policy_document" "demo_apigw_allow_manage_resources" {
version = "2012-10-17"
statement {
actions = [
"logs:DescribeLogGroups",
"logs:DescribeLogStreams",
"logs:GetLogEvents",
"logs:FilterLogEvents"
]
resources = [
"*"
]
}
statement {
actions = [
"logs:CreateLogDelivery",
"logs:PutResourcePolicy",
"logs:UpdateLogDelivery",
"logs:DeleteLogDelivery",
"logs:CreateLogGroup",
"logs:DescribeResourcePolicies",
"logs:GetLogDelivery",
"logs:ListLogDeliveries"
]
resources = [
"*"
]
}
}
data "aws_iam_policy_document" "demo_apigw_allow_assume_role" {
version = "2012-10-17"
statement {
effect = "Allow"
actions = [
"sts:AssumeRole"]
principals {
type = "Service"
identifiers = ["apigateway.amazonaws.com"]
}
}
}
resource "aws_iam_role_policy" "demo_apigw_allow_manage_resources" {
policy = data.aws_iam_policy_document.demo_apigw_allow_manage_resources.json
role = aws_iam_role.demo_apigw_cloudwatch_role.id
name = var.demo-apigw-manage-resources_policy_name
}
resource "aws_iam_role" "demo_apigw_cloudwatch_role" {
name = "demo_apigw_cloudwatch_role"
tags = var.custom_tags
assume_role_policy = data.aws_iam_policy_document.demo_apigw_allow_assume_role.json
}
You can route CW logs (aws_cloudwatch_log_group) to /aws/vendedlogs/* and it will resolve issue. Or create aws_api_gateway_account

terraform not working with remote exec for tpl script

I have a simple aws ec2 instance as below
resource "aws_instance" "App01" {
##ami = "ami-846144f8"
ami = "${data.aws_ami.aws_linux.id}"
instance_type = "t1.micro"
subnet_id = "${aws_subnet.public_1a.id}"
associate_public_ip_address = true
vpc_security_group_ids = ["${aws_security_group.web_server.id}","${aws_security_group.allow_ssh.id}"]
key_name = "key"
provisioner "remote-exec"{
inline = ["${template_file.bootstrap.rendered}"]
}
tags {
Name = "App01"
}
}
data "aws_ami" "aws_linux" {
most_recent = true
filter {
name = "name"
values = ["amzn2-ami-*-x86_64-gp2"]
}
filter {
name = "virtualization-type"
values = ["hvm"]
}
filter {
name = "owner-alias"
values = ["amazon"]
}
}
resource "template_file" "bootstrap" {
template = "${file("bootstrap.tpl")}"
vars {
app01ip = "${aws_instance.App01.private_ip}"
app02ip = "${aws_instance.App02.private_ip}"
DBandMQip = "${aws_instance.DBandMQ.private_ip}"
}
}
This is my tbl script
#!/bin/bash -xe
# install necessary items like ansible and
sudo yum-config-manager --enable epel
sudo amazon-linux-extras install ansible2
echo "${app01ip} App01" > /etc/hosts
echo "${app02ip} App02" > /etc/hosts
echo "${DBandMQip} DBandMQ" > /etc/hosts
I keep getting a
Error: Error asking for user input: 1 error(s) occurred:
* Cycle: aws_instance.App01, template_file.bootstrap
I believe its coming from the resource portion for remote-exec but I am unsure whats wrong because it looks fine to me. Anyone has any idea what I am doing wrong?

Terraform (provider AWS) - Auto Scaling group doesn't take effect on a launch template change

Unable to make launch template work with ASGs while using launch templates, it works with Launch Configuration using a small hack i.e by interpolating the launch configuration name in ASG resource but it doesn't work with launch templates.
ASG uses the latest version to launch new instances but doesn't change anything w.r.t to pre-running instances inspite of a change in launch template.
I understand that this is sort of expected but do we have any workaround to make launch templates work with ASG or we need to stick to launch configuration itself?
TF code snippet -
resource "aws_launch_template" "lc_ec2" {
image_id = "${var.ami_id}"
instance_type = "${var.app_instance_type}"
key_name = "${var.orgname}_${var.environ}_kp"
vpc_security_group_ids = ["${aws_security_group.sg_ec2.id}"]
user_data = "${base64encode(var.userdata)}"
block_device_mappings {
device_name = "/dev/xvdv"
ebs {
volume_size = 15
}
}
iam_instance_profile {
name = "${var.orgname}_${var.environ}_profile"
}
lifecycle {
create_before_destroy = true
}
tag_specifications {
resource_type = "instance"
tags = "${merge(map("Name", format("%s-%s-lc-ec2", var.orgname, var.environ)), var.tags)}"
}
tag_specifications {
resource_type = "volume"
tags = "${merge(map("Name", format("%s-%s-lc-ec2-volume", var.orgname, var.environ)), var.tags)}"
}
tags = "${merge(map("Name", format("%s-%s-lc-ec2", var.orgname, var.environ)), var.tags)}"
}
resource "aws_autoscaling_group" "asg_ec2" {
name = "${var.orgname}-${var.environ}-asg-ec2-${aws_launch_template.lc_ec2.name}"
vpc_zone_identifier = ["${data.aws_subnet.private.*.id}"]
min_size = 1
desired_capacity = 1
max_size = 1
target_group_arns = ["${aws_lb_target_group.alb_tg.arn}"]
default_cooldown= 100
health_check_grace_period = 100
termination_policies = ["ClosestToNextInstanceHour", "NewestInstance"]
health_check_type="ELB"
launch_template = {
id = "${aws_launch_template.lc_ec2.id}"
version = "$$Latest"
}
lifecycle {
create_before_destroy = true
}
tags = [
{
key = "Name"
value = "${var.orgname}"
propagate_at_launch = true
},
{
key = "Environ"
value = "${var.environ}"
propagate_at_launch = true
}
]
}
There is one hack to achieve this.
AWS CloudFormation supports rolling updates of an Autoscaling group.
Since Terraform supports a cloudformation stack resource, you can define your ASG as a cloudformation stack with an update policy. However, CloudFormation does not support the $$Latest tag for launch template version, so you will have to parameterize the version and take the input value from the latest_version attribute of the launch template resource created in your terraform configuration file.

Resources