I want to develop a single Terraform module to deploy my resources, with the resources being stored in separate YAML files. For example:
# resource_group_a.yml
name: "ResourceGroupA"
location: "westus"
# resource_group_b.yml
name: "ResourceGroupB"
location: "norwayeast"
And the following Terraform module:
# deploy/main.tf
variable source_file {
type = string # Path to a YAML file
}
locals {
rg = yamldecode(file(var.source_file))
}
resource "azurerm_resource_group" "rg" {
name = local.rg.name
location = local.rg.location
}
I can deploy the resource groups with:
terraform apply -var="source_file=resource_group_a.yml"
terraform apply -var="source_file=resource_group_b.yml"
But then I run into 2 problems, due to the state that Terraform keeps about my infrastructure:
If I deploy Resource Group A, it deletes Resource Group B and vice-versa.
If I manually remove the .tfstate file prior to running apply, and the resource group already exists, I get an error:
A resource with the ID "/..." already exists - to be managed via Terraform
this resource needs to be imported into the State.
with azurerm_resource_group.rg,
on main.tf line 8 in resource "azurerm_resource_group" "rg"
I can import the resource into my state with
terraform import azurerm_resource_group.reg "/..."
But it's a long file and there may be multiple resources that I need to import.
So my questions are:
How to keep the state separate between the two resource groups?
How to automatically import existing resources when I run terraform apply?
How to keep the state separate between the two resource groups?
I recommend using Terraform Workspaces for this, which will give you separate state files, each with an associated workspace name.
How to automatically import existing resources when I run terraform
apply?
That's not currently possible. There are some third-party tools out there like Terraformer for accomplishing automated imports, but in my experience they don't work very well, or they never support all the resource types you need. Even then they wouldn't import resources automatically every time you run terraform apply.
Related
I'm trying to import 2 existing azure resources (1 vnet and 1 resource group).
If I add the following to my main.tf
resource "azurerm_virtual_network" "my-vnet" {
}
and run terraform import azurerm_virtual_network.my-vnet /subscriptions/11111111/resourceGroups/my-resource-group/providers/Microsoft.Network/virtualNetworks/my-vnet-01
Everything works perfectly, the terraform.tfstate file gets updated with the vnet information etc.
If I then add the following to the main.tf
resource "azurem_resource_group" "my-net-rsg" {
}
and run terraform import azurem_resource_group.my-net-rsg /subscriptions/11111111/resourceGroups/my-net-rsg
I get the following error
If I then remove
resource "azurem_resource_group" "my-net-rsg" {
}
and run the same terraform import azurem_resource_group.my-net-rsg /subscriptions/11111111/resourceGroups/my-net-rsg I get an error (quite rightly & expectedly) saying
Before importing this resource, please create its configuration in the root module. For example:
resource "azurem_resource_group" "my-net-rsg" {
# (resource arguments)
}
Even if I literally copy and paste that output back into my main.tf file I get the same error from the screen shot above.
I've tried only having the resource group config
I've tried having the resource config first before the vnet
I've tried a fresh terraform init in a different directory and walking through the process again.
Every time if I have the resource group in the main.tf it complains about the lock file.
I was just going to delete the question as it is such a simple problem/solution, but because the error is so non-descript as to the cause this might be helpful to someone
Apparently
Is terraform for you have a typo in your resource name.
resource "azurem_resource_group" should be (in this case) resource "azurerm_resource_group"
I have a diagnostic setting configured on my master db. As shown below in my main.tf
resource "azurerm_monitor_diagnostic_setting" "main" {
name = "Diagnostic Settings - Master"
target_resource_id = "${azurerm_mssql_server.main.id}/databases/master"
log_analytics_workspace_id = azurerm_log_analytics_workspace.main.id
log {
category = "SQLSecurityAuditEvents"
enabled = true
retention_policy {
enabled = false
}
}
metric {
category = "AllMetrics"
retention_policy {
enabled = false
}
}
lifecycle {
ignore_changes = [log, metric]
}
}
If I don't delete it before in the resource group before I run the Terraform, I get the error:
Diagnostic Settings - Master" already exists - to be managed via
Terraform this resource needs to be imported into the State
I know that if I delete the SQL Server the diagnostic setting remains - but I don't know why that is a problem with Terraform. I have also noticed that it is in my tfplan.
What could be the problem?
If I don't delete it before in the resource group before I run the
Terraform, I get the error:
Diagnostic Settings - Master" already exists - to be managed via Terraform this resource needs to be imported into the State
I know that if I delete the SQL Server the diagnostic setting remains but I don't know why that is a problem with Terraform.
If you have created the resource in Azure from a different way (i.e. Portal/Templates/CLI/Powershell), that means Terraform is not aware of resource already existing in Azure. So, during Terraform Plan, it shows you the plan what will be created from what you have written in main.tf. But when you run Terraform Apply the azurerm provider checks the resources names with the existing resources of the same resource providers and result in giving an error that it already exists and needs to be imported to be managed by Terraform.
Also if you have created everything from Terraform then doing a Terraform destroy deletes all the resources present on the main.tf.
Well, it's in the .tfplan and also it's in main.tf - so it's imported, right ?
If you mention the resource and its details in main.tf and .tfplan, it doesn't mean that you have imported the resource or Terraform gets aware of the resource. Terraform is only aware of the resources that are stored in the Terraform state file i.e. .tfstate.
So , to overcome the error that you get without deleting the resource from Portal, you will have to add the resource in the main.tf as you have already done and then use Terraform import command to import the Azure resource to Terraform State file like below:
terraform import azurerm_monitor_diagnostic_setting.example "{resourceID}|{DiagnosticsSettingsName}"
So, for you it will be like:
terraform import azurerm_monitor_diagnostic_setting.main "/subscriptions/<SubscriptionID>/resourceGroups/<ResourceGroupName>/providers/Microsoft.Sql/servers/<SQLServerName>/databases/master|Diagnostic Settings - Master"
After the Import is done, any changes you make from Terraform to that resource will get reflected in portal as well and you will be able to destroy the resource from terraform as well.
Our terraform remote state file in Azure has been completely destroyed and we're now faced with the challenge of recreating the state file from scratch. My option is to use the Terraform import command, using the following simple syntax:
terraform import <Terraform Resource Name>.<Resource Label> <Azure Resource ID>
To import the existing resource group for example, I will create the following configuration in a main.tf file.
provider "azurerm" {
version="1.39.0"
}
# create resource group
resource "azurerm_resource_group" "rg"{
name = "rg-terraform"
location = "uksouth"
}
Now, the problem I have is as follows:
When the existing Azure resources were originally created, they were assigned names that used an extremely complex naming convention, with some characters even being randomly generated. To further compound matters, they were all unique and there are hundreds of them. All would have been rosy if they were assigned a simplistic name like "main", as is used commonly in a lot of Terraform examples, but unfortunately, that's not the case.
My question therefore, is this:
When putting together my main.tf configuration file to be used for the Import, is it an absolute requirement that my "Resource Label" (given in my Import command) has to match the original "Resource Label" name from when the resource was created?
If it is a mandatory requirement, is there any way I could retrieve the original "Resource Label" from Azure in the same way that I can for instance obtain the "Azure Resource ID" from the Azure Portal or even an Az CLI query?
How can I ensure any child resources such as Subnets are included in the Import, without having to trawl manually through the Azure Portal to identify each one of them?
No, absolutely not. Choose whatever you want.
No, Azure generally does not know about this label, it is something terraform internal.
Unfortunately you need to import each and every resource manually and separately.
Have you made absolutely sure the current state file is lost? The storage location was not versioned? Does no developer still have a local copy of the state file laying around?
I'm trying to create Azure resources from Azure web site CLI, but strangely it throws the error below after run "terraform apply". It wasn't like this before.
Error: A resource with the ID "/subscriptions/certdab-as441-4a670-bsda-2456437/resourceGroups/commapi" already exists - to be managed via Terraform this resource needs to be imported into the State. Please see the resource documentation for "azurerm_resource_group" for more information.
on terra.tf line 14, in resource "azurerm_resource_group" "rg1":
14: resource "azurerm_resource_group" "rg1" {
Terraform code is below;
provider "azurerm" {
version = "=2.20.0"
features {}
subscription_id = "c413asdasdasdadsasda1c77"
tenant_id = "da956asdasdadsadsasd25a535"
}
resource "azurerm_resource_group" "rg" {
name = "commapi"
location = "West Europe"
}
resource "azurerm_resource_group" "rg1" {
name = "commapi"
location = "West Europe"
}
Regarding to their web site, if a resource group is alredy created, we should import it. But I don't get it how?
They said I should import like below, should I add this line to my terraform code?
Or should I run this import command for every Azure tool?
terraform import azurerm_resource_group.example /subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/group1
So for example if I already created 10 Azure resources before and if I add 11.th tool to my Terraform code, should I run this "import" command for each that 10 resources which already created before? That's so weird.
How can I create these resources?
Edit:
If I try again, throws the error below;
Error: A resource with the ID "/subscriptions/asdd-asde1-4asd-bsda-asasd/resourceGroups/commerceapi/providers
/Microsoft.ApiManagement/service/commapi-apim" already exists - to be managed via Terraform this resource needs to be imported into the State. Please see the resource documentation for "azurerm_api_management" for more information.
on terra.tf line 65, in resource "azurerm_api_management" "apm":
65: resource "azurerm_api_management" "apm" {
Thanks!
Sample 2 :
For example when I create the API, it throws the error below;
Plan: 1 to add, 0 to change, 0 to destroy.
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
azurerm_api_management.apm: Creating...
Error: A resource with the ID "/subscriptions/c4112313-123-123-123-1c77/resourceGroups/testapi/providers/Microsoft.ApiManagement/service/api-apim" already exists - to be managed via Terraform this resource needs to be imported into the State. Please see the resource documentation for "azurerm_api_management" for more information.
on commerceApi.tf line 67, in resource "azurerm_api_management" "apm":
67: resource "azurerm_api_management" "apm" {
After that, when I try to import command, this time it throws the error below;
PS C:\Users\set\Desktop\Terraform\Test_Commerceapi> terraform import azurerm_api_management.apm /subscriptions/c4asdb-234e1-23-234a-23424337/resourceGroups/testapi
azurerm_api_management.apm: Importing from ID "/subscriptions/c4234324-234-234-23-4347/resourceGroups/testapi"...
azurerm_api_management.apm: Import prepared!
Prepared azurerm_api_management for import
azurerm_api_management.apm: Refreshing state... [id=/subscriptions/c4234324-23-4234-234324-77/resourceGroups/testapi]
Error: making Read request on API Management Service "" (Resource Group "testapi"): apimanagement.ServiceClient#Get: Invalid input: autorest/validation: validation failed: parameter=serviceName constraint=MinLength value="" details: value length must be greater than or equal to 1
If you want to bring existing infrastructure under Terraform management, you can use Import.
For example, import an existing resource group.
declare the resource in the .tf file like this.
resource "azurerm_resource_group" "rg" {
}
Then run terraform import azurerm_resource_group.rg /subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/group1 to import the existing resource group.
Then you can edit the Terraform file and add the name and location of the existing resource to the Resource Group. After this, you are ready to use the resource. Read how to Import an Existing Azure Resource in Terraform for more details.
Please note that Terraform import can only import resources into the state. It does not generate configuration. If you just intend to create new resources relying on existing resources, you could use data sources.
Data sources allow data to be fetched or computed for use elsewhere in
Terraform configuration. Use of data sources allows a Terraform
configuration to make use of information defined outside of Terraform,
or defined by another separate Terraform configuration.
For example, use Data Source: azurerm_resource_group to access information about an existing Resource Group. You can create new resources within the existing VNet.
data "azurerm_resource_group" "example" {
name = "existing"
}
output "id" {
value = data.azurerm_resource_group.example.id
}
I have a hosted zone in the main.tf:
provider "aws" {
region = "us-east-1"
}
resource "aws_route53_zone" "zone" {
}
I then can import an existing resource and use its parameters in other resources:
terraform import aws_route53_zone.zone <ZoneId>
Inspecting the state file I see the parameters are all there, including the domain name. But when I want to apply it it says that name is not found:
Error: aws_route53_zone.zone: "name": required field is not set
I don't want to specify the name in the .tf file as it would decrease the portability of my .tf, but specifying a placeholder would change the hosted zone itself.
Is there a way to ignore parameters for imported resources or specify them as "leave as-is"?
I could add a variable and populate that from the state file for every terraform calls, but I'm hoping for something simpler.
When you import a resource, Terraform doesn't (yet) automatically generate Terraform code for you and instead you must write the resource and then check the plan.
Normally the pattern would be to create a skeleton resource like you've done, import the resource and then fill out any required fields, run a plan, then adjust your resource configuration so that it doesn't make any unwanted changes.
From then on Terraform will be able to manage the resource as normal, applying any changes you make to the configuration or reverting changes done outside of Terraform back to how they are done in your Terraform code.