I Need some help in configuring AWS backup vaults in multiple AWS accounts using terraform. I'm able to create backup vaults in 2 accounts with specific plan and schedule. but i cant see the backedup data on the destination account. Here's the code which i'm using.
resource "aws_backup_vault" "backup-vault" {
provider = aws.source-account
name = var.backup-vault-name
kms_key_arn = aws_kms_key.backup-key.arn
}
resource "aws_backup_vault" "diff-account-vault" {
provider = aws.crossbackup
name = var.cross-account-vault-name
kms_key_arn = aws_kms_key.backup-key.arn
}
resource "aws_backup_plan" "backup-plan" {
name = var.backup-plan-name
rule {
rule_name = "some-rule"
target_vault_name = aws_backup_vault.backup-vault.name
schedule = "cron(0 17-23 * * ? *)"
copy_action {
destination_vault_arn = aws_backup_vault.diff-account-vault.arn
}
}
}
resource "aws_backup_selection" "tag" {
name = "some-backup-selection-name"
iam_role_arn = aws_iam_role.aws-backup-service-role.arn
plan_id = aws_backup_plan.backup-plan.id
selection_tag {
type = var.selection-type
key = var.key
value = var.value
}
}
resource "aws_backup_vault_policy" "organization-policy" {
backup_vault_name = aws_backup_vault.diff-account-vault.name
provider = aws.crossbackup
policy = <<POLICY
{
"Version":"2012-10-17",
"Statement":[
{
"Effect":"Allow",
"Action":"backup:CopyIntoBackupVault",
"Resource":"*",
"Principal":"*",
"Condition":{
"StringEquals":{
"aws:PrincipalOrgID":[
"Organization-ID"
]
}
}
}
]
}
POLICY
}
Related
I have seen similar questions but the answers there rather addressed the formatting or workarounds which weren't that "clean". I will try to summarize my issue and hopefully can get some lean/clean solution. Thank you in advance guys!
I am creating AKS namespaces via Kubernetes provider in Terraform. Since I have 3 clusters, I want to be able to control which provider should be used to create the namespace. e.g. dev / prod
Folder structure
.terraform
├───modules
└───namespace.tf
└───module_providers.tf
└───main-deploy
└───main.tf
└───main_provider.tf
My Module // namespace.tf
# Create Namespace
resource "kubernetes_namespace" "namespace-appteam" {
metadata {
annotations = {
name = var.application_name
}
labels = {
appname = var.application_name
}
name = var.application_name
}
}
My main.tf file
module "appteam-test" {
source = "../modules/aks-module"
application_name = "dev-app"
providers = {
kubernetes.dev = kubernetes.dev
kubernetes.prod = kubernetes.prod
}
}
Now since I have passed 2 providers in the main.tf module block! How do I control that the resource I am creating in namespace.tf file should use Dev or Prod provider? In short, how does the module know which resource to create with what provider if several are passed.
Note: I have required_providers defined in the module_providers.tf and providers in main_provider.tf folder.
module_provider.tf
terraform {
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = "3.20.0"
}
azuread = {
source = "hashicorp/azuread"
version = "2.27.0"
}
kubernetes = {
source = "hashicorp/kubernetes"
version = "2.12.1"
}
}
}
Main_provider.tf
provider "azuread" {
alias = "AD"
}
provider "azurerm" {
alias = "default"
features {}
}
provider "kubernetes" {
alias = "dev"
}
provider "kubernetes" {
alias = "prod"
}
You need to add alias to all your providers.
provider "kubernetes" {
alias = "dev"
}
provider "kubernetes" {
alias = "stage"
}
In your main.tf file. Pass providers in module like following:
providers = {
kubernetes.dev = kubernetes.dev
kubernetes.stage = kubernetes.stage
}
Now in the module_provider.tf file. You need to pass configuration_aliases
kubernetes = {
source = "hashicorp/kubernetes"
version = "2.12.1"
configuration_aliases = [ kubernetes.dev, kubernetes.stage ]
}
Once all the configurations are in place. You can specify the provider explicitly for resources you want. Your namespace.tf file will look like
resource "kubernetes_namespace" "namespace-appteam-1" {
provider = kubernetes.dev
metadata {
annotations = {
name = var.application_name
}
labels = {
appname = var.application_name
}
name = var.application_name
}
}
resource "kubernetes_namespace" "namespace-appteam-2" {
provider = kubernetes.stage
metadata {
annotations = {
name = var.application_name
}
labels = {
appname = var.application_name
}
name = var.application_name
}
}
Below is a small snippet of a set of terraform script I'm trying to build. The goal is to define an IAM policy that will be attached to a new IAM role that I will create.
My problem is I'm trying to use the environment tag that I've defined in my AWS provider's default_tags block but I'm not sure how. The goal is to pull the environment value as part of the S3 prefix in the IAM policy document instead of having it hard coded.
Is there a way to do this?
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "4.19.0"
}
}
required_version = ">=1.2.3"
}
provider "aws" {
default_tags {
tags = {
Environment = "dev"
Application = "myapp"
Terraform = "true"
}
}
}
data "aws_iam_policy_document" "this" {
statement {
sid = "S3BucketAccess"
actions = "s3:*"
resources = [
"${data.aws_s3_bucket.this.arn}/dev"
]
}
}
data "aws_s3_bucket" "this" {
bucket = "myBucket"
}
Notice
A solution without code duplication is to use aws_default_tags:
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "4.19.0"
}
}
required_version = ">=1.2.3"
}
provider "aws" {
default_tags {
tags = {
Environment = "dev"
Application = "myapp"
Terraform = "true"
}
}
}
# Get the default tags from the provider
data "aws_default_tags" "my_tags" {}
data "aws_iam_policy_document" "this" {
statement {
sid = "S3BucketAccess"
actions = "s3:*"
resources = ["${data.aws_s3_bucket.this.arn}/${data.aws_default_tags.my_tags.tags.Environment}/*"
]
}
}
The solution is to use locals.
Here's how the final solution looks like
# New locals block
locals {
common_tags = {
Environment = "dev"
Application = "myapp"
Terraform = "true"
}
}
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "4.19.0"
}
}
required_version = ">=1.2.3"
}
provider "aws" {
# Reference common_tags from locals
default_tags {
tags = local.common_tags
}
}
data "aws_iam_policy_document" "this" {
# In the resources statement, I replaced
# "dev" prefix with Environment tag value using locals
statement {
sid = "S3BucketAccess"
actions = "s3:*"
resources = ["${data.aws_s3_bucket.this.arn}/${local.common_tags.Environment}/*"
]
}
}
data "aws_s3_bucket" "this" {
bucket = "myBucket"
}
I need some assistance to undertand the various forms of logging in to Databricks. I am using Terraform to provision Azure Databricks
I would like to know the difference in the two codes below
When i use option 1, i get the error as shown
Option 1:
required_providers {
azuread = "~> 1.0"
azurerm = "~> 2.0"
azuredevops = { source = "registry.terraform.io/microsoft/azuredevops", version = "~> 0.0" }
databricks = { source = "registry.terraform.io/databrickslabs/databricks", version = "~> 0.0" }
}
}
provider "random" {}
provider "azuread" {
tenant_id = var.project.arm.tenant.id
client_id = var.project.arm.client.id
client_secret = var.secret.arm.client.secret
}
provider "databricks" {
host = azurerm_databricks_workspace.db-workspace.workspace_url
azure_use_msi = true
}
resource "azurerm_databricks_workspace" "db-workspace" {
name = module.names-db-workspace.environment.databricks_workspace.name_unique
resource_group_name = module.resourcegroup.resource_group.name
location = module.resourcegroup.resource_group.location
sku = "premium"
public_network_access_enabled = true
custom_parameters {
no_public_ip = true
virtual_network_id = module.virtualnetwork["centralus"].virtual_network.self.id
public_subnet_name = module.virtualnetwork["centralus"].virtual_network.subnets["db-sub-1-public"].name
private_subnet_name = module.virtualnetwork["centralus"].virtual_network.subnets["db-sub-2-private"].name
public_subnet_network_security_group_association_id = module.virtualnetwork["centralus"].virtual_network.nsgs.associations.subnets["databricks-public-nsg-db-sub-1-public"].id
private_subnet_network_security_group_association_id = module.virtualnetwork["centralus"].virtual_network.nsgs.associations.subnets["databricks-private-nsg-db-sub-2-private"].id
}
tags = local.tags
}
Databricks Cluster Creation
resource "databricks_cluster" "dbcselfservice" {
cluster_name = format("adb-cluster-%s-%s", var.project.name, var.project.environment.name)
spark_version = var.spark_version
node_type_id = var.node_type_id
autotermination_minutes = 20
autoscale {
min_workers = 1
max_workers = 7
}
azure_attributes {
availability = "SPOT_AZURE"
first_on_demand = 1
spot_bid_max_price = 100
}
depends_on = [
azurerm_databricks_workspace.db-workspace
]
}
Databricks Workspace RBAC Permission
resource "databricks_group" "db-group" {
display_name = format("adb-users-%s", var.project.name)
allow_cluster_create = true
allow_instance_pool_create = true
depends_on = [
resource.azurerm_databricks_workspace.db-workspace
]
}
resource "databricks_user" "dbuser" {
count = length(local.display_name)
display_name = local.display_name[count.index]
user_name = local.user_name[count.index]
workspace_access = true
depends_on = [
resource.azurerm_databricks_workspace.db-workspace
]
}
Adding Members to Databricks Admin Group
resource "databricks_group_member" "i-am-admin" {
for_each = toset(local.email_address)
group_id = data.databricks_group.admins.id
member_id = databricks_user.dbuser[index(local.email_address, each.key)].id
depends_on = [
resource.azurerm_databricks_workspace.db-workspace
]
}
data "databricks_group" "admins" {
display_name = "admins"
depends_on = [
# resource.databricks_cluster.dbcselfservice,
resource.azurerm_databricks_workspace.db-workspace
]
}
The error that i get while TF apply is below :
Error: User not authorized
with databricks_user.dbuser[1],
on resources.adb.tf line 80, in resource "databricks_user" "dbuser":
80: resource "databricks_user" "dbuser"{
Error: User not authorized
with databricks_user.dbuser[0],
on resources.adb.tf line 80, in resource "databricks_user" "dbuser":
80: resource "databricks_user" "dbuser"{
Error: cannot refresh AAD token: adal:Refresh request failed. Status Code = '500'. Response body: {"error":"server_error", "error_description":"Internal server error"} Endpoint http://169.254.169.254/metadata/identity/oauth2/token?api-version=2018-02-01&resource=https%3A%2F%2Fmanagement.core.windows.net%2F
with databricks_group.db-group,
on resources.adb.tf line 80, in resource "databricks_group" "db-group":
71: resource "databricks_group" "db-group"{
Is the error coming because of this block below ?
provider "databricks" {
host = azurerm_databricks_workspace.db-workspace.workspace_url
azure_use_msi = true
}
I just need to login automatically when i click on the URL from the portal. So what shall i use for that? And why do we need to provide two times databricks providers, once under required_providers and again in provider "databricks"?
I have seen if i don't provide the second provider i get the error :
"authentication is not configured for provider"
As mentioned in the comments , If you are using Azure CLI authentication i.e. az login using your username and password , then you can use the below code :
terraform {
required_providers {
databricks = {
source = "databrickslabs/databricks"
version = "0.3.11"
}
}
}
provider "azurerm" {
features {}
}
provider "databricks" {
host = azurerm_databricks_workspace.example.workspace_url
}
resource "azurerm_databricks_workspace" "example" {
name = "DBW-ansuman"
resource_group_name = azurerm_resource_group.example.name
location = azurerm_resource_group.example.location
sku = "premium"
managed_resource_group_name = "ansuman-DBW-managed-without-lb"
public_network_access_enabled = true
custom_parameters {
no_public_ip = true
public_subnet_name = azurerm_subnet.public.name
private_subnet_name = azurerm_subnet.private.name
virtual_network_id = azurerm_virtual_network.example.id
public_subnet_network_security_group_association_id = azurerm_subnet_network_security_group_association.public.id
private_subnet_network_security_group_association_id = azurerm_subnet_network_security_group_association.private.id
}
tags = {
Environment = "Production"
Pricing = "Standard"
}
}
data "databricks_node_type" "smallest" {
local_disk = true
depends_on = [
azurerm_databricks_workspace.example
]
}
data "databricks_spark_version" "latest_lts" {
long_term_support = true
depends_on = [
azurerm_databricks_workspace.example
]
}
resource "databricks_cluster" "dbcselfservice" {
cluster_name = "Shared Autoscaling"
spark_version = data.databricks_spark_version.latest_lts.id
node_type_id = data.databricks_node_type.smallest.id
autotermination_minutes = 20
autoscale {
min_workers = 1
max_workers = 7
}
azure_attributes {
availability = "SPOT_AZURE"
first_on_demand = 1
spot_bid_max_price = 100
}
depends_on = [
azurerm_databricks_workspace.example
]
}
resource "databricks_group" "db-group" {
display_name = "adb-users-admin"
allow_cluster_create = true
allow_instance_pool_create = true
depends_on = [
resource.azurerm_databricks_workspace.example
]
}
resource "databricks_user" "dbuser" {
display_name = "Rahul Sharma"
user_name = "example#contoso.com"
workspace_access = true
depends_on = [
resource.azurerm_databricks_workspace.example
]
}
resource "databricks_group_member" "i-am-admin" {
group_id = databricks_group.db-group.id
member_id = databricks_user.dbuser.id
depends_on = [
resource.azurerm_databricks_workspace.example
]
}
Output:
If you are using Service Principal as authentication , then you can use something like below:
terraform {
required_providers {
databricks = {
source = "databrickslabs/databricks"
version = "0.3.11"
}
}
}
provider "azurerm" {
subscription_id = "948d4068-xxxx-xxxx-xxxx-e00a844e059b"
tenant_id = "72f988bf-xxxx-xxxx-xxxx-2d7cd011db47"
client_id = "f6a2f33d-xxxx-xxxx-xxxx-d713a1bb37c0"
client_secret = "inl7Q~Gvdxxxx-xxxx-xxxxyaGPF3uSoL"
features {}
}
provider "databricks" {
host = azurerm_databricks_workspace.example.workspace_url
azure_client_id = "f6a2f33d-xxxx-xxxx-xxxx-d713a1bb37c0"
azure_client_secret = "inl7Q~xxxx-xxxx-xxxxg6ntiyaGPF3uSoL"
azure_tenant_id = "72f988bf-xxxx-xxxx-xxxx-2d7cd011db47"
}
resource "azurerm_databricks_workspace" "example" {
name = "DBW-ansuman"
resource_group_name = azurerm_resource_group.example.name
location = azurerm_resource_group.example.location
sku = "premium"
managed_resource_group_name = "ansuman-DBW-managed-without-lb"
public_network_access_enabled = true
custom_parameters {
no_public_ip = true
public_subnet_name = azurerm_subnet.public.name
private_subnet_name = azurerm_subnet.private.name
virtual_network_id = azurerm_virtual_network.example.id
public_subnet_network_security_group_association_id = azurerm_subnet_network_security_group_association.public.id
private_subnet_network_security_group_association_id = azurerm_subnet_network_security_group_association.private.id
}
tags = {
Environment = "Production"
Pricing = "Standard"
}
}
data "databricks_node_type" "smallest" {
local_disk = true
depends_on = [
azurerm_databricks_workspace.example
]
}
data "databricks_spark_version" "latest_lts" {
long_term_support = true
depends_on = [
azurerm_databricks_workspace.example
]
}
resource "databricks_cluster" "dbcselfservice" {
cluster_name = "Shared Autoscaling"
spark_version = data.databricks_spark_version.latest_lts.id
node_type_id = data.databricks_node_type.smallest.id
autotermination_minutes = 20
autoscale {
min_workers = 1
max_workers = 7
}
azure_attributes {
availability = "SPOT_AZURE"
first_on_demand = 1
spot_bid_max_price = 100
}
depends_on = [
azurerm_databricks_workspace.example
]
}
resource "databricks_group" "db-group" {
display_name = "adb-users-admin"
allow_cluster_create = true
allow_instance_pool_create = true
depends_on = [
resource.azurerm_databricks_workspace.example
]
}
resource "databricks_user" "dbuser" {
display_name = "Rahul Sharma"
user_name = "example#contoso.com"
workspace_access = true
depends_on = [
resource.azurerm_databricks_workspace.example
]
}
resource "databricks_group_member" "i-am-admin" {
group_id = databricks_group.db-group.id
member_id = databricks_user.dbuser.id
depends_on = [
resource.azurerm_databricks_workspace.example
]
}
And why do we need to provide
two times databricks providers, once under required_providers and
again in provider "databricks"?
The required_providers is used to download and initialize the required providers from the source i.e. Terraform Registry . But the Provider Block is used for further configuration of that downloaded provider like describing client_id, features block etc. which can be used for authentication or other configuration.
The azure_use_msi option is primarily intended for use from CI/CD pipelines that are executed on machines with managed identity assigned to them. All possible authentication options are described in the documenation, but simplest way is to use authentication via Azure CLI, so you just need to leave host parameter in the provider block. If you don't have Azure CLI on that machine, you can use combination of host + personal access token instead.
if you're running that code from the machine with assigned managed identity, then you need to make sure that this identity is either added into workspace, or it has Contributor access to it - see Azure Databricks documentation for more details.
I created a webserver infrastructure with terraform (v0.12.21) in Gcloud to deploy a lot of websites.
I created a persistent volume claim for each deploy (1GB each):
I used this code to create them:
resource "kubernetes_persistent_volume_claim" "wordpress_volumeclaim" {
for_each = var.wordpress_site
metadata {
name = "wordpress-volumeclaim-${terraform.workspace}-${each.value.name}"
namespace = "default"
}
spec {
access_modes = ["ReadWriteOnce"]
resources {
requests = {
storage = each.value.disk
resource_policies = google_compute_resource_policy.policy.name
}
}
}
}
resource "kubernetes_deployment" "wordpress" {
for_each = var.wordpress_site
metadata {
name = each.value.name
labels = { app = each.value.name }
}
spec {
replicas = 1
selector {
match_labels = { app = each.value.name }
}
template {
metadata {
labels = { app = each.value.name }
}
spec {
volume {
name = "wordpress-persistent-storage-${terraform.workspace}-${each.value.name}"
persistent_volume_claim {
claim_name = "wordpress-volumeclaim-${terraform.workspace}-${each.value.name}"
}
}
[...]
But now I need to backup all these disks, and my best idea is using the Gcloud snapshot functionallity, and it must be dynamic, as the creation of these disks are dynamic.
First of all, I created a Snapshot policy:
resource "google_compute_resource_policy" "policy" {
name = "my-resource-policy"
region = "zone-region-here"
project = var.project
snapshot_schedule_policy {
schedule {
daily_schedule {
days_in_cycle = 1
start_time = "04:00"
}
}
retention_policy {
max_retention_days = 7
on_source_disk_delete = "KEEP_AUTO_SNAPSHOTS"
}
}
}
And now I want to add it to my persistent volumen claim. But I dont know how, because this line is not working at all:
resource_policies = google_compute_resource_policy.policy.name
All my tries resulted in errors. Could you help me here?
I'm trying to create a service account secret in order to populate a secret with the token:
resource "kubernetes_service_account" "k8s-api-token" {
metadata {
namespace = "${var.whatever_namespace}"
name = "api-service-account"
}
secret {
name = "api-service-account-secret"
}
}
resource "kubernetes_secret" "k8s-api-token" {
metadata {
namespace = "${var.whatever_namespace}"
name = "${kubernetes_service_account.k8s-api-token.metadata.0.name}-secret"
annotations = {
"kubernetes.io/service-account.name" = "${kubernetes_service_account.k8s-api-token.metadata.0.name}"
}
}
type = "kubernetes.io/service-account-token"
}
data "kubernetes_secret" "k8s-api-token" {
depends_on = ["kubernetes_secret.k8s-api-token"]
metadata {
namespace = "${var.whatever_namespace}"
name = "${kubernetes_secret.k8s-api-token.metadata.0.name}"
}
}
resource "kubernetes_secret" "whatever-secrets" {
depends_on = ["kubernetes_secret.k8s-api-token"]
metadata {
name = "botfront-secrets"
namespace = "${var.whatever_namespace}"
}
data = {
K8S_API = "${data.kubernetes_secret.k8s-api-token.data.token}"
}
}
But it gives me an error:
Resource 'data.kubernetes_secret.k8s-api-token' does not have attribute 'data.token' for variable 'data.kubernetes_secret.k8s-api-token.data.token'
I can verify the secret is created, but even running terraform state show kubernetes_secret.k8s_api_token doesn't return anything
What am I doing wrong?
The solution is to use a lookup:
K8S_API = ${lookup(data.kubernetes_secret.k8s-api-token-data.data, "token","")}
Source: http://blog.crashtest-security.com/resource-does-not-have-attribute