We are trying to spin up an azure subscription using terraform. But unable to find any exact code which delivers successful subscription creation. Below is the brief description of the possible scenarios we have tried executing.
We used the below code and executed it and the role we had was
Account Owner Role -But the code is successful but we aren't able to
see the subscription in the portal. Link for the code:
https://github.com/aztfmod/terraform-azurerm-caf/tree/5.4.8/modules/subscriptions
Below is the next code we tested with again Account Owner role - But
the error message is below. Link for the code: https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/subscription#example-usage---creating-a-new-alias-and-subscription-for-a-microsoft-customer-account
Error Message: creating new Subscription (Alias "XXXX7"):
subscription.AliasClient#Create: Failure sending request:
StatusCode=0 -- Original Error: Code="InvalidSubCreationScope"
Message="Not a valid subscription creation scope", with
azurerm_subscription.testtf, on main.tf line 31, in resource
"azurerm_subscription" "testtf": 31: resouce
"azurerm_subscription" "testtf" {
Another code we tried executing - But its destroying the older
subscription. Link for the code is same:
https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/subscription#example-usage---creating-a-new-alias-and-subscription-for-a-microsoft-customer-account
Request any suggestions/help at the earliest for the same so can help us.
Thanks in Advance!
Error Message: creating new Subscription (Alias "XXXX7"): subscription.AliasClient#Create: Failure sending request: StatusCode=0
-- Original Error: Code="InvalidSubCreationScope" Message="Not a valid subscription creation scope", with azurerm_subscription.testtf, on main.tf line 31, in resource "azurerm_subscription" "testtf": 31: resouce "azurerm_subscription" "testtf" {
As I have mentioned in comments , The above issue was with the billing scope which was being used for creating the subscription . Instead of Name you should use Billing account No. and Enrollment no. in the billing_account_name & enrollment_account_name Arguments like below :
data "azurerm_billing_enrollment_account_scope" "example" {
billing_account_name = "1234567890"
enrollment_account_name = "0123456"
}
resource "azurerm_subscription" "example" {
subscription_name = "My Example EA Subscription"
billing_scope_id = data.azurerm_billing_enrollment_account_scope.example.id
}
Note: Please make sure that you are using all the correct details which are can be found from EA portal.
i get this recently and the problem was that the identity which is used to call the subscription creation in Terraform does not have the subscription creator role of the new api. my spn had the subscription role of the legacy api.
to grant the subscription role to the spn, use this link:
focus on this section: Assign the subscription creator role to the SPN
https://learn.microsoft.com/en-us/azure/cost-management-billing/manage/assign-roles-azure-service-principals
https://learn.microsoft.com/en-us/rest/api/billing/2019-10-01-preview/enrollment-account-role-assignments/put?tabs=HTTP
Related
I am trying to new resource group in azure using terraform but i am getting following error
Error checking for presence of existing resource group:
resources.GroupsClient#Get: Failure responding to request:
StatusCode=403 -- Original Error: autorest/azure: Service returned an
error. Status=403 Code="AuthorizationFailed" Message="The client
'xxxxx' with object id 'xxxxx' does not have authorization to perform
action 'Microsoft.Resources/subscriptions/resourcegroups/read' over
scope '/subscriptions/***/resourcegroups/stage-group' or the scope
is invalid. If access was recently granted, please refresh your
credentials.
This is code sample
terraform {
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = "=2.7.0"
}
}
}
provider "azurerm" {
features {}
}
#Create resource group
resource "azurerm_resource_group" "resource_group" {
name = "stage-group"
location = "eastus"
}
I am running this through github actions and passing ARM_CLIENT_ID,ARM_CLIENT_SECRET, ARM_SUBSCRIPTION_ID ,ARM_TENANT_ID . I dont have problem when running the same for existing resource.
You have to assign a proper role to your Service Principal at a higher scope - Subscription or Management group level, with the rights to create Resource Groups (Microsoft.Resources/subscriptions/resourceGroups/write).
Most of the time, Contributor is a good fit but you can look for more granular roles depending on your needs.
In the terraform documentation for google_pubsub_subscription, it mentions having a oidc_token property under push_configuration allows push-subscriptions to utilise service account, but has no mention on how to use service account for pull subscriptions.
How can explicitly set the service account to be used during the creation of pull pubsub subscriptions?
I tried adding oidc_token block, but it did not worked as it does not expect that block directly.
Scenario:
I have a service account that has access to pubsub topics (and necessary permissions to attach subscriptions to it) in Project_A
I want to create subscriptions to those topics in Project_B in terraform.
I need to explicitly use this service account in terraform, so i can create subscriptions at Project_B to topics of Project_A
google_cloudfunctions_function resource for example, has field called service_account_email for setting service account. But there is no for google_pubsub_subscription resource, for pull subscriptions in the documentation.
Actually the service account (service_account_email)is to be specified inside the oidc_token section (you had to go further in the doc :)
below a working example of a push sub with service account and audience (optional)
variable "project" {
type = string
default = "<YOUR_PROJECT_ID>"
}
resource "google_pubsub_topic" "example" {
project = var.project
name = "example-topic"
}
resource "google_pubsub_subscription" "example" {
project = var.project
name = "example-push-subscription-with-auth"
topic = google_pubsub_topic.example.name
push_config {
push_endpoint = "https://example.com/push"
oidc_token {
service_account_email = "${var.project}#appspot.gserviceaccount.com"
audience = "https://example.com/push"
}
}
}
I'm adding another answer, because the question has changes a lot after the different comments.
Here is the solution with the hypothesis that a topic topic-sof is already created in a different project than the subscription one.
I created a service account (SA) on the subscription project, I called it stackoverflow-playground-sa and gave it only the Pub/Sub Editor role as you can see in the screenshot below.
I gave the SA the the Pub/Sub Subscriber role on the topic topic-sof s show in the screenshot below.
If you don't do this step: you will get this error at terraform apply
Error: Error creating Subscription: googleapi: Error 403: User not authorized to perform this action
Of course I did the 2 first steps of roles assignments with a user with enough permissions on both projects
I created a json key file for my SA and downloaded under /path/to/my/key/stackoverflow-playground-sa.json
I authenticated as my SA using
export GOOGLE_APPLICATION_CREDENTIALS=/path/to/my/key/stackoverflow-playground-sa.json
So it can be used by terraform to create the subscription.
Here is the terraform configuration to create the subscription
variable "subscription_project" {
type = string
default = "<YOUR_SUBSCRIPTION_PROJECT>"
}
variable "topic_project" {
type = string
default = "<YOUR_TOPIC_PROJECT>"
}
resource "google_pubsub_subscription" "pull-subscription-of-topic-in-another-project" {
project = var.subscription_project
name = "pull-subscription-of-topic-in-another-project"
topic = "projects/${var.topic_project}/topics/topic-sof"
}
Run (apply) my terraform and the subscription pull-subscription-of-topic-in-another-project is created and attached to the topic topic-sof in the other project.
I published a message to topic-sof using the web ui.
Still authenticated as stackoverflow-playground-sa thanks to step 4, I pull the message (using gcloud in my treminal) et voilĂ message received:
To summarise : there is no service account to specify in your terraform configuration for this requirement. The service account and its key is set outside terraform (step 1 to 4) so that terraform process can be authenticated and authorised to create the resources configured (the subscription in our case).
Besides using a service account key as in step 4 is a bad practice, security wise. An alternative is to use
gcloud auth application-default login
That will let you set the default credentials of the user of your choice. Given that user has the roles I set for the SA.
I try to run terraform with azurerm provider initialized for service principal authorization, with the following in providers.tf (with service principal that has owner access on the subscription and works in other code):
provider "azurerm" {
features {}
subscription_id = "SSSSSSSS-SSSS-SSSS-SSSS-SSSSSSSSSSSS"
client_id = "AAAAAAAA-AAAA-AAAA-AAAA-AAAAAAAAAAAA"
tenant_id = "<MY_TENANT_ID>"
client_secret = "<MY_CLIENT_SECRET>"
}
However, when I run terraform plan or terraform apply, this fails reporting totally different client ID:
Error: Error ensuring Resource Providers are registered.
[.... some lines of useless examples ....]
Original Error: Cannnot register providers: Microsoft.ServiceFabricMesh, Microsoft.ManagedServices, Microsoft.DesktopVirtualization. Errors were: Cannot register provider Microsoft.ServiceFabricMesh with Azure Resource Manager: resources.ProvidersClient#Register: Failure responding to request: StatusCode=403 -- Original Error: autorest/azure: Service returned an error. Status=403 Code="AuthorizationFailed" Message="The client 'BBBBBBBB-BBBB-BBBB-BBBB-BBBBBBBBBBBB' with object id 'BBBBBBBB-BBBB-BBBB-BBBB-BBBBBBBBBBBB' does not have authorization to perform action '
Microsoft.ServiceFabricMesh/register/action' over scope '/subscriptions/SSSSSSSS-SSSS-SSSS-SSSS-SSSSSSSSSSSS' or the scope is invalid. If access was recently granted, please refresh your credentials.".
In other words, while I tell terraform to use service principal "AAAAAAAA-AAAA-AAAA-AAAA-AAAAAAAAAAAA", it uses "BBBBBBBB-BBBB-BBBB-BBBB-BBBBBBBBBBBB" instead.
It is obvious to say that nothing in my code refers a service principal with id "BBBBBBBB-BBBB-BBBB-BBBB-BBBBBBBBBBBB".
Setting ARM_* environment variables does not have any effect on this behavior.
Any idea how could that happen?
Update: I'm on terraform 0.14.3 and azurerm 2.41. Also tried azurerm 2.32 - it behaves the same.
First of all, you can output the client id to make sure that you are using the correct service principal that has the appropriate authorization scope on the subscription and tenant you're working with.
data "azurerm_client_config" "current" {
}
output "account_id" {
value = data.azurerm_client_config.current.client_id
}
Also, If you're using service principal credentials, try re-generating your secret key or your service principal. As a workaround, you also could skip the provider registration as
provider "azurerm" {
skip_provider_registration = true
}
At last, try to upgrade your terraform and azurerm provider for a better experience. It might be some bug in some versions, refer to this1 and this2.
I solved this - but still do not know why incorrect service principal id was used by terraform.
Once I registered the groups manually, the issue with 'BBBBBBBB...' principal id happened once more when terraform tried to create a resource in not existing group (there was a typo in RG name). I still can't understand why terraform reported that weird service principal issue instead of missed group, but the issue has gone now.
Thanks everyone who helped me to narrow it!
What specific changes need to be made to the syntax below in order for the terraform azurerm provider to be able to authenticate the service principal that will be created using the following code?
The Problem
A Second Terraform module needs to authenticate to Azure through the azurerm provider with a client_id and client_secret that is created programatically during an earlier, separate process.
The provider block in the Second Terraform module looks like:
provider "azurerm" {
subscription_id = var.subscriptionId
client_id = var.clientId
client_secret = var.clientSecret
tenant_id = var.tenantId
}
The problem arises when the correct values whcih we validated from the earlier preceding process are not accepted as the var.clientId and the var.clientSecret in the provider code block above.
How the Service Principal is Created:
The client_id and client_secret to be used to authenticate to the Second Terraform module are currently created by a First Terraform module which includes the following:
resource "azuread_application" "appReg" {
name = var.appName
}
resource "azuread_service_principal" "example-sp" {
application_id = azuread_application.appReg.application_id
}
resource "azuread_service_principal_password" "example-sp_pwd" {
service_principal_id = azuread_service_principal.example-sp.id
value = "long-random-string"
end_date = "2021-06-02T01:02:03Z"
}
data "azurerm_subscription" "thisSubscription" {
subscription_id = var.subscriptionId
}
resource "azurerm_role_assignment" "example-sp_role_assignment" {
scope = data.azurerm_subscription.thisSubscription.id
role_definition_name = "Contributor"
principal_id = azuread_service_principal.example-sp.id
}
resource "azuread_application_app_role" "example-role" {
application_object_id = azuread_application.appReg.id
allowed_member_types = ["User", "Application"]
description = "Admins can manage roles and perform all task actions"
display_name = "Admin"
is_enabled = true
value = "administer"
}
Terraform reports Apply complete after the above First module is run, and we are also able to confirm in the Azure Portal that the correct Active Directory has a new app registration with name var.appName and with ID equal to what we find in the First modules tfstate file.
The Error Message:
When Terraform tries to apply the Second module using the Service Principal ID and Secret created by the First module, the following error is thrown:
Error:
Error building account:
Error getting authenticated object ID:
Error listing Service Principals:
autorest.DetailedError{
Original:adal.tokenRefreshError{
message:"adal: Refresh request failed.
Status Code = '400'.
Response body: {
\"error\":\"unauthorized_client\",
\"error_description\":\"AADSTS700016:
Application with identifier 'correct-app-id' was not found in the directory 'the-right-ad-id'.
This can happen if the application has not been installed by the administrator of the tenant or consented to by any user in the tenant.
You may have sent your authentication request to the wrong tenant.\\r\\n
Trace ID: some-trace-id\\r\\n
Correlation ID: correlation-id-redacted\\r\\n
Timestamp: 2020-12-31 19:02:19Z\",
\"error_codes\":[700016],
\"timestamp\":\"2020-12-31 19:02:19Z\",
\"trace_id\":\"some-trace-id\",
\"correlation_id\":\"correlation-id-redacted\",
\"error_uri\":\"https://login.microsoftonline.com/error?code=700016\"
}",
resp:(*http.Response)(0xc000ac2000)},
PackageType:"azure.BearerAuthorizer",
Method:"WithAuthorization",
StatusCode:400,
Message:"Failed to refresh the Token for request to https://graph.windows.net/the-right-ad-id/servicePrincipals?%24filter=appId+eq+%27correct-app-id%27&api-version=1.6",
ServiceError:[]uint8(nil),
Response:(*http.Response)(0xc000ac2000)
}
The error message does not seem helpful because we validated that the app is registered with the AAD instance.
How can we resolve this problem and programmatically create a client_id and client_secret that will be accepted and usable by the Second module?
As I see there is no problem with your Terraform code. It should work fine. But you got the error that the application was not found in the tenant. So what you need to do is to check if the tenant Id is really right in the second module.
I had the same issue on Deployment Agents for Terraform on Kubernetes. Several types of error can appear when the memory or the CPU is not large enough.
Below are the Terraform recommendations: https://www.terraform.io/docs/enterprise/before-installing/index.html
You have to be careful with the deployment infrastructures that mutualises the resources (K8s, Hypervisor Pool etc.)when several Terraform deployments are in parallel it causes somewhat random errors.
Terraform which does not stop, API AZure / AWS error, tfstate lock etc.
I am facing an issue with following Azure API.
GET https://management.azure.com/{resourceUri}/providers/microsoft.insights/metricDefinitions?api-version=2018-01-01
I am trying to access the above API using my Subscription ID in place of resourceUri. But it is throwing the below error:
{
"error": {
"code": "MissingSubscription",
"message": "The request did not have a provided subscription. All requests must have an associated subscription Id."
}
}
I have checked in Azure Portal, everything seems fine. The Subscription ID is correct one. The user account have owner permissions. Still getting the error. Not sure of the reason.
Can someone help me out with the most common and basic reasons of this error and steps to resolve this one. Already I have spent couple of hours debugging this one, but no luck.
I have reproduced your issue, the resourceUri should be the Resource ID instead of the Subscription ID.
Use Subscription ID :
Use Resource ID(in my sample, I use a web app resource id, you could find it in the portal-> your web app -> Properties):