Azure pipeline - terratest - ERROR: Please run 'az login' to setup account - azure

Question

i'm facing a (it seams) recurent pbm in Azure Pipeline to run terratest.
While resources are well created the destroyed, when I call an azure.ResourceGroupExists function (or whatever else azure.xxx function) i have the following error :
--- FAIL: TestTerraform_RM_resource_group (102.30s)
resourcegroup.go:15:
Error Trace: resourcegroup.go:15
RM_resource_group_test.go:108
Error: Received unexpected error:
Invoking Azure CLI failed with the following error: ERROR: Please run 'az login' to setup account.
Test: TestTerraform_RM_resource_group
FAIL
Regarding some forum, It seems to be some configuration issue, and I follow all these recomanded configuratoion :
set environments variables for terraform :
-- ARM_CLIENT_ID
-- ARM_CLIENT_SECRET
-- ARM_SUBSCRIPTION_ID
-- ARM_TENANT_ID
set the az login in AzureCli task outside the go task for terratest, as it seems that terratest needs 2 differents authentifications : (using service principal client id for this az login)
For Assert tests, needs the ARM_CLIENT authentification
for Exists tests, needs the Service connection authentification
here the link I follow :
https://github.com/gruntwork-io/terratest/issues/454
https://github.com/gruntwork-io/terratest/tree/master/examples/azure#review-environment-variables
https://github.com/gruntwork-io/terratest/blob/master/modules/environment/envvar.go
https://blog.jcorioland.io/archives/2019/09/25/terraform-microsoft-azure-ci-docker-azure-pipeline.html
bellow my pipeline code, where the TF_VAR_ARM_CLIENT_SECRET is a secret variable of the pipeline
runOnce:
deploy:
steps:
- checkout: self
- task: ms-devlabs.custom-terraform-tasks.custom-terraform-installer-task.TerraformInstaller#0
displayName: 'Install Terraform $(TERRAFORM_VERSION)'
inputs:
terraformVersion: $(TERRAFORM_VERSION)
- task: GoTool#0
displayName: 'Use Go $(GOVERSION)'
inputs:
version: $(GOVERSION)
goPath: $(GOPATH)
goBin: $(GOBIN)
- task: Go#0
displayName: 'Install Go Terratest module'
inputs:
command: get
arguments: '$(TF_LOG) github.com/gruntwork-io/terratest/modules/terraform'
- task: Go#0
displayName: 'Install Go Assert module'
inputs:
command: get
arguments: '$(TF_LOG) github.com/stretchr/testify/assert'
- task: Go#0
displayName: 'Install Go Terratest Azure module'
inputs:
command: get
arguments: '$(TF_LOG) github.com/gruntwork-io/terratest/modules/azure'
- task: Go#0
displayName: 'Install Go hashicorp/terraform-json module'
inputs:
command: get
arguments: '$(TF_LOG) github.com/hashicorp/terraform-json'
- task: Go#0
displayName: 'Install Go azure-sdk-for-go module'
inputs:
command: get
arguments: '$(TF_LOG) github.com/Azure/azure-sdk-for-go'
- task: AzureCLI#2
displayName: Azure CLI
inputs:
azureSubscription: $(serviceConnection)
scriptType: ps
scriptLocation: inlineScript
inlineScript: |
az login --service-principal --username $(TF_VAR_ARM_CLIENT_ID) --password $(TF_VAR_ARM_CLIENT_SECRET) --tenant 'f5ff14e7-93c8-49f7-9706-7beea059bd32'
# Go test command
- task: Go#0
displayName: 'Run Go terratest for resource_Modules'
inputs:
command: test
arguments: '$(TF_LOG) $(pathToTerraformRootModule)\resource_group\'
env:
ARM_CLIENT_SECRET: $(TF_VAR_ARM_CLIENT_SECRET) #pipeline secret variable
ARM_CLIENT_ID: $(TF_VAR_ARM_CLIENT_ID)
ARM_SUBSCRIPTION_ID: $(TF_VAR_ARM_SUBSCRIPTION_ID)
ARM_TENANT_ID: $(TF_VAR_ARM_TENANT_ID)
TF_VAR_SERVICE_PRINCIPAL_ID: $(TF_VAR_ARM_CLIENT_ID)
TF_VAR_SERVICE_PRINCIPAL_SECRET: $(TF_VAR_ARM_CLIENT_ID)
resource_group_name: $(storageAccountResourceGroup)
storage_account_name: $(storageAccount)
container_name: $(stateBlobContainer)
key: '$(MODULE)-$(TF_VAR_APPLICATION)-${{ parameters.Environment }}.tfstate'
Bellow my go terratest code :
package RM_resource_group_Test
import (
"testing"
"os"
"github.com/gruntwork-io/terratest/modules/azure"
"github.com/gruntwork-io/terratest/modules/terraform"
"github.com/stretchr/testify/assert"
)
var (
globalBackendConf = make(map[string]interface{})
globalEnvVars = make(map[string]string)
)
func TestTerraform_RM_resource_group(t *testing.T) {
t.Parallel()
// terraform Directory
fixtureFolder := "./"
// input value
inputStage := "demo_we"
inputEnvironment := "DEMO"
inputApplication := "DEMO"
// expected value
expectedName := "z-adf-ftnd-shrd-dm-ew1-rgp42"
// getting enVars from environment variables
ARM_CLIENT_ID := os.Getenv("ARM_CLIENT_ID")
ARM_CLIENT_SECRET := os.Getenv("ARM_CLIENT_SECRET")
ARM_SUBSCRIPTION_ID := os.Getenv("ARM_SUBSCRIPTION_ID")
ARM_TENANT_ID := os.Getenv("ARM_TENANT_ID")
if ARM_CLIENT_ID != "" {
globalEnvVars["ARM_USE_MSI"] = "false"
globalEnvVars["ARM_CLIENT_ID"] = ARM_CLIENT_ID
globalEnvVars["ARM_CLIENT_SECRET"] = ARM_CLIENT_SECRET
globalEnvVars["ARM_SUBSCRIPTION_ID"] = ARM_SUBSCRIPTION_ID
globalEnvVars["ARM_TENANT_ID"] = ARM_TENANT_ID
}
// getting backend vars from environment variables
resource_group_name := os.Getenv("resource_group_name")
storage_account_name := os.Getenv("storage_account_name")
container_name := os.Getenv("container_name")
key := os.Getenv("key")
if resource_group_name != "" {
globalBackendConf["use_msi"] = false
globalBackendConf["resource_group_name"] = resource_group_name
globalBackendConf["storage_account_name"] = storage_account_name
globalBackendConf["container_name"] = container_name
globalBackendConf["key"] = key
}
// User Terratest to deploy the infrastructure
terraformOptions := terraform.WithDefaultRetryableErrors(t, &terraform.Options{
// The path to where our Terraform code is located
TerraformDir: fixtureFolder,
// Variables to pass to our Terraform code using -var options
Vars: map[string]interface{}{
"STAGE": inputStage,
"ENVIRONMENT": inputEnvironment,
"APPLICATION" : inputApplication,
},
EnvVars: globalEnvVars,
// backend values to set when initialziing Terraform
BackendConfig: globalBackendConf,
// Disable colors in Terraform commands so its easier to parse stdout/stderr
NoColor: true,
})
// website::tag::4::Clean up resources with "terraform destroy". Using "defer" runs the command at the end of the test, whether the test succeeds or fails.
// At the end of the test, run `terraform destroy` to clean up any resources that were created
defer terraform.Destroy(t, terraformOptions)
// website::tag::2::Run "terraform init" and "terraform apply".
// This will run `terraform init` and `terraform apply` and fail the test if there are any errors
terraform.InitAndApply(t, terraformOptions)
actualName := terraform.Output(t, terraformOptions, "tested_name")
actualReaderName := terraform.Output(t, terraformOptions, "tested_readerName")
assert.Equal(t, expectedName, actualName)
assert.Equal(t, expectedName, actualReaderName)
subscriptionID := terraform.Output(t, terraformOptions, "current_subscription_id")
exists := azure.ResourceGroupExists(t, expectedName, subscriptionID)
assert.True(t, exists, "Resource group does not exist")
}
I'm sure I miss something in passing my parameters, as always I have the following error, after creating and destroying resources in Azure :
--- FAIL: TestTerraform_RM_resource_group (90.75s)
resourcegroup.go:15:
Error Trace: resourcegroup.go:15
RM_resource_group_test.go:108
Error: Received unexpected error:
Invoking Azure CLI failed with the following error: ERROR: Please run 'az login' to setup account.
Test: TestTerraform_RM_resource_group
please, help.

Answer 1

and thank-you for answering..
As I figure out earlier, it was a configuration mistake and, after having made some deep excavations on Go Terratest Azure module, I've found these lines that gives all the explanations :
https://github.com/gruntwork-io/terratest/blob/master/modules/azure/authorizer.go#L11
leading to
https://learn.microsoft.com/en-us/azure/developer/go/azure-sdk-authorization#use-environment-based-authentication
So I change my pipeline to this :
# Go test command
- task: Go#0
displayName: 'Run Go terratest for resource_Modules'
inputs:
command: test
arguments: '$(TF_LOG) $(pathToTerraformRootModule)\...'
env:
ARM_SUBSCRIPTION_ID: $(TF_VAR_ARM_SUBSCRIPTION_ID)
AZURE_CLIENT_ID: $(TF_VAR_ARM_CLIENT_ID)
AZURE_TENANT_ID: $(TF_VAR_ARM_TENANT_ID)
AZURE_CLIENT_SECRET: $(TF_VAR_ARM_CLIENT_SECRET)
resource_group_name: $(storageAccountResourceGroup)
storage_account_name: $(storageAccount)
container_name: $(stateBlobContainer)
key: '$(MODULE)-$(TF_VAR_APPLICATION)-${{ parameters.Environment }}.tfstate'
And my Go code to this (regarding the envVariables use) :
// getting enVars from environment variables
ARM_CLIENT_ID := os.Getenv("AZURE_CLIENT_ID")
ARM_CLIENT_SECRET := os.Getenv("AZURE_CLIENT_SECRET")
ARM_TENANT_ID := os.Getenv("AZURE_TENANT_ID")
ARM_SUBSCRIPTION_ID := os.Getenv("ARM_SUBSCRIPTION_ID")
// creating globalEnVars for terraform call through Terratest
if ARM_CLIENT_ID != "" {
//globalEnvVars["ARM_USE_MSI"] = "true"
globalEnvVars["ARM_CLIENT_ID"] = ARM_CLIENT_ID
globalEnvVars["ARM_CLIENT_SECRET"] = ARM_CLIENT_SECRET
globalEnvVars["ARM_SUBSCRIPTION_ID"] = ARM_SUBSCRIPTION_ID
globalEnvVars["ARM_TENANT_ID"] = ARM_TENANT_ID
}
// getting backend vars from environment variables
resource_group_name := os.Getenv("resource_group_name")
storage_account_name := os.Getenv("storage_account_name")
container_name := os.Getenv("container_name")
key := os.Getenv("key")
// creating globalBackendConf for terraform call through Terratest
if resource_group_name != "" {
//globalBackendConf["use_msi"] = true
globalBackendConf["resource_group_name"] = resource_group_name
globalBackendConf["storage_account_name"] = storage_account_name
globalBackendConf["container_name"] = container_name
globalBackendConf["key"] = key
}
// User Terratest to deploy the infrastructure
terraformOptions := terraform.WithDefaultRetryableErrors(t, &terraform.Options{
// website::tag::1::Set the path to the Terraform code that will be tested.
// The path to where our Terraform code is located
TerraformDir: fixtureFolder,
// Variables to pass to our Terraform code using -var options
Vars: map[string]interface{}{
"STAGE": inputStage,
"ENVIRONMENT": inputEnvironment,
"APPLICATION" : inputApplication,
//"configuration" : inputConfiguration,
},
// globalvariables for user account
EnvVars: globalEnvVars,
// backend values to set when initialziing Terraform
BackendConfig: globalBackendConf,
// Disable colors in Terraform commands so its easier to parse stdout/stderr
NoColor: true,
})
And all goes right !
Hopes this could help others.
Thanks again.
[EDIT] To be more explicit :
Go and Terraform uses two differents methods for Azure authentification.
** Terraform authentification is explained bellow :
https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/guides/service_principal_client_secret#configuring-the-service-principal-in-terraform
** Go authentification is explained bellow :
https://learn.microsoft.com/en-us/azure/developer/go/azure-sdk-authorization#use-environment-based-authentication
** Terratest is using both authentification methods regarding the work it has to be done :
azure existences tests uses Go azure authentification :
https://github.com/gruntwork-io/terratest/blob/master/modules/azure/authorizer.go#L11
terraform commands uses terraform authentification :
https://github.com/gruntwork-io/terratest/blob/0d654bd2ab781a52e495f61230cf892dfba9731b/modules/terraform/cmd.go#L12
https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/guides/service_principal_client_secret#configuring-the-service-principal-in-terraform
so both authentification methods have to be implemented