I am facing issues from azure metadata service when I am trying to run my terraform plan.
Things I tried.
removing ~/.azure folder , re-tried az login from browser.
removed ~/.Identity folder as well , but still the same.
Also added client_id , client_secret , etc from provider block but no luck.
Error: retrieving environments from Azure MetaData service: Get "https:///metadata/endpoints?api-version=2019-05-01": http: no Host in request URL
on main.tf line 1, in provider "azurerm":
1: provider "azurerm" {
Any help would be greatly appreciated. I tried already to check if this was raised before but no luck.
I was able to fix this issue by doing 2 things mentioned below.
remove the ~/.azure and ~/.Identitiy folder from local.
export ARM_MSI_ENDPOINT=false
Once you do that you should now be able to access the metadata API again for authentication to azure.
Explanation of MSI flag is here terraform provider MSI use
I wanted to use service principal to authenticate and somehow I got this enabled in provider.
Related
I'm trying to use DefaultAzureCredentials to authenticate my Azure function against Azure Service Bus. In my azure function azure-func-service-bus, I call to Azure Service Bus
servicebus_client = ServiceBusClient(
fully_qualified_namespace=MY_SERVICE_BUS_NAMESPACE_NAME+".servicebus.windows.net",
credential=DefaultAzureCredential(additionally_allowed_tenants=['*'])
)
I created and pushed Docker container to ACR. When I run the container locally for testing outside of Azure, it does not know what permissions to use.
az acr login --name acr01
docker push acr01.azurecr.io/azure-func-service-bus:v1
docker pull acr01.azurecr.io/azure-func-service-bus:v1
docker run -it --rm -p 8080:80 acr01.azurecr.io/azure-func-service-bus:v1
but got the following error.
DefaultAzureCredential failed to retrieve a token from the included credentials.
Attempted credentials:
EnvironmentCredential: EnvironmentCredential authentication unavailable. Environment variables are not fully configured.
Visit https://aka.ms/azsdk/python/identity/environmentcredential/troubleshoot to troubleshoot.this issue.
ManagedIdentityCredential: ManagedIdentityCredential authentication unavailable, no response from the IMDS endpoint.
SharedTokenCacheCredential: SharedTokenCacheCredential authentication unavailable. No accounts were found in the cache.
VisualStudioCodeCredential: Failed to get Azure user details from Visual Studio Code.
AzureCliCredential: Azure CLI not found on path
AzurePowerShellCredential: PowerShell is not installed
To mitigate this issue, please refer to the troubleshooting guidelines here at https://aka.ms/azsdk/python/identity/defaultazurecredential/troubleshoot.
Unexpected error occurred (ClientAuthenticationError('DefaultAzureCredential failed to retrieve a token from the included credentials.\nAttempted credentials:\n\tEnvironmentCredential: EnvironmentCredential authentication unavailable. Environment variables are not fully configured.\nVisit https://aka.ms/azsdk/python/identity/environmentcredential/troubleshoot to troubleshoot.this issue.\n\tManagedIdentityCredential: ManagedIdentityCredential authentication unavailable, no response from the IMDS endpoint.\n\tSharedTokenCacheCredential: SharedTokenCacheCredential authentication unavailable. No accounts were found in the cache.\n\tVisualStudioCodeCredential: Failed to get Azure user details from Visual Studio Code.\n\tAzureCliCredential: Azure CLI not found on path\n\tAzurePowerShellCredential: PowerShell is not installed\nTo mitigate this issue, please refer to the troubleshooting guidelines here at https://aka.ms/azsdk/python/identity/defaultazurecredential/troubleshoot.')). Handler shutting down.
I'm missing a key piece of the puzzle. How can I handle this?
When the Azure Function runs in Azure, it's configured to support ManagedIdentityCredential. For your case I'd recommend trying to configure EnvironmentCredential to test locally.
You can find the details in the link, but the short version is:
Create a service principle (Docs) and give it the needed access
Run the container with extra Environment Variables:
AZURE_TENANT_ID: service principal's Tenant ID
AZURE_CLIENT_ID: service principal's AppId
AZURE_CLIENT_SECRET: service principle's password
I'd recommend using a .env file to make this easier, but be sure it doesn't get checked in anywhere.
FYI If your account doesn't use MFA, you can instead use the variables AZURE_USERNAME and AZURE_PASSWORD. But then you've put your username and password in a file or your terminal history which is concerning. Admittedly the service principal has the same problem, but you can more easily mitigate that with minimizing it's access and regularly rolling the secret.
P.S. If you're using Visual Studio for making your Azure Function you should be able to use something like: EnvironmentCredentialExample to automate setting up and using the needed .env file.
I'm working on a Terraform project in which I setup several Azure resources.
One of these Azure resources is a service principal (linked to an app registration) which I use to deploy my Terraform code in a CI/CD pipeline via Github actions.
When developing locally, I use az login to authenticate, but occasionally I'm receiving an error for the Terraform app service principal. Most of the times, when I re-run terraform apply the error is not raised. Sometimes the error persists for several terraform apply calls.
Error: Retrieving Application with object ID "fe2b93b7-e26c-402c-ab4f-87e3695c1f45" with module.app_registrations.azuread_application.terraform_app on ../modules/app_registrations/terraform.tf line 56, in resource "azuread_application" "terraform_app": 56: resource "azuread_application" "terraform_app" { ApplicationsClient.BaseClient.Get(): Get "https://graph.microsoft.com/beta/b859b851-97d8-4dc2-bf56-f2a5bc5c494b/applications/fe2b93b7-e26c-402c-ab4f-87e3695c1f45": http: RoundTripper implementation (*retryablehttp.RoundTripper) returned a nil *Response with a nil error
I'm having a hard time to understand why I'm able to generate this service principal using my user credentials via Terraform, but am receiving this retrieval error when refreshing the state of the same service principal. When I'm deploying the Terraform code via Github Actions, which uses the service principal to authenticate, this retrieval error is never raised.
Anyone can point me in the right direction?
I've already added the Application administrator role to my user credentials and added the Application.ReadWrite.All role to the Terraform service principal. Above error when refreshing the state using user credentials via az login persists.
Error: authentication is not configured for provider. Please configure it through one of the following options: 1. DATABRICKS_HOST + DATABRICKS_TOKEN environment variables. 2. host + token provider arguments. 3. azure_databricks_workspace_id + AZ CLI authentication. 4. azure_databricks_workspace_id + azure_client_id + azure_client_secret + azure_tenant_id for Azure Service Principal authentication. 5. Run databricks configure --token that will create ~/.databrickscfg file.
Please check and Advice about this error
You can try with below workaround given in this github discussion to troubleshoot your issue
Overall recommendation is to also separate workspace creation (azurerm provider) and for workspace provisioning (databricks provider).
the other workaround is to have an empty ~/.databrickscfg file, so locals block might be avoided. not ideal, but will work..
you can also managed to work around it by using locals to pre-configure related resource names and then reference those when building the workspace resource id in the provider config
and databricks provider resources all have a depends_on block with the databricks workspace
I have been using the below to successfully create a back-end state file for terraform in Azure storage, but for some reason its stopped working. I've recycled passwords for the storage, trying both keys and get the same error every-time
backend.tf
terraform {
backend "azurerm" {
storage_account_name = "terraformstorage"
resource_group_name = "automation"
container_name = "terraform"
key = "testautomation.terraform.tfstate"
access_key = "<storage key>"
}
}
Error returned
terraform init
Initializing the backend...
Successfully configured the backend "azurerm"! Terraform will automatically
use this backend unless the backend configuration changes.
Error refreshing state: storage: service returned error: StatusCode=403, ErrorCode=AuthenticationFailed, ErrorMessage=Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
RequestId:665e0067-b01e-007a-6084-97da67000000
Time:2018-12-19T10:18:18.7148241Z, RequestInitiated=Wed, 19 Dec 2018 10:18:18 GMT, RequestId=665e0067-b01e-007a-6084-97da67000000, API Version=, QueryParameterName=, QueryParameterValue=
Any ideas what im doing wrong?
What worked for me is to delete the local .terraform folder and try again.
Another problem can be time resolution.
I experienced those problems as well, tried all the above mentioned steps, but nothing helped.
What happened on my system (Windows 10, WSL2) was, that WSL lost its time sync and I was hours apart. This behaviour is described in https://github.com/microsoft/WSL/issues/4245.
For me it helped to
get the appropriate time in WSL (sudo hwclock -s) and
to reboot WSL
Hope, this will help others too.
Here are few suggestions:
Run: terraform init -reconfigure.
Confirm your "terraform/backend" credentials.
In case your Terraform contains some "azurerm_storage_account/network_rules" to allow certain IP addresses, or make sure you're connected to the right VPN network.
If above won't work, run TF_LOG=TRACE terraform init to debug further.
Please ensure you've been authenticated properly to Azure Cloud.
If you're running Terraform externally, re-run: az login.
If you're running Terraform on the instance, you can use managed identities, or by defining the following environmental variables:
ARM_USE_MSI=true
ARM_SUBSCRIPTION_ID=xxx-yyy-zzz
ARM_TENANT_ID=xxx-yyy-zzz
or just run az login --identity, then assign the right role (azurerm_role_assignment, e.g. "Contributor") and appropriate policies (azurerm_policy_definition).
See also:
Azure Active Directory Provider: Authenticating using Managed Service Identity.
Unable to programmatically get the keys for Azure Storage Account.
There should a .terraform directory , where you are running the terraform init command from.
Remove .terraform or move it to Someotehr name. Next time terraform init runs , it will recreate that directory with new init.
I have enabled Managed Service Identities on an App Service. However, my WebJobs seem unable to access the keys.
They report:
Tried the following 3 methods to get an access token, but none of them worked.
Parameters: Connectionstring: [No connection string specified], Resource: https://vault.azure.net, Authority: . Exception Message: Tried to get token using Managed Service Identity. Unable to connect to the Managed Service Identity (MSI) endpoint. Please check that you are running on an Azure resource that has MSI setup.
Parameters: Connectionstring: [No connection string specified], Resource: https://vault.azure.net, Authority: https://login.microsoftonline.com/common. Exception Message: Tried to get token using Active Directory Integrated Authentication. Access token could not be acquired. password_required_for_managed_user: Password is required for managed user
Parameters: Connectionstring: [No connection string specified], Resource: https://vault.azure.net, Authority: . Exception Message: Tried to get token using Azure CLI. Access token could not be acquired. 'az' is not recognized as an internal or external command,
Kudo does not show any MSI_ environmental variables.
How is this supposed to work? This is an existing App Service Plan.
The AppAuthentication library leverages an internal endpoint in App Service that receives the tokens on your site's behalf. This endpoint is non-static and therefore is set to an environment variable. After activating MSI for your site through ARM, your site will need to be restarted to get two new Environment Variables set in it:
MSI_ENDPOINT and MSI_SECRET
The presence of these variables are essential to the MSI feature working properly during runtime as the AppAuthentication library uses them to get the authorization token. The error message reflects this:
Exception Message: Tried to get token using Managed Service Identity. Unable to connect to the Managed Service Identity (MSI) endpoint. Please check that you are running on an Azure resource that has MSI setup.
If these variables are absent, you might need to restart the site.
https://learn.microsoft.com/en-us/azure/app-service/app-service-managed-service-identity
If the environment variables are set and you still see the same error, the article above has a code sample showing how to send requests to that endpoint manually.
public static async Task<HttpResponseMessage> GetToken(string resource, string apiversion) {
HttpClient client = new HttpClient();
client.DefaultRequestHeaders.Add("Secret", Environment.GetEnvironmentVariable("MSI_SECRET"));
return await client.GetAsync(String.Format("{0}/?resource={1}&api-version={2}", Environment.GetEnvironmentVariable("MSI_ENDPOINT"), resource, apiversion));
}
I would try that and see what kind of response I get back.
I just solved this issue when trying to use MSI with a Function app, though I already had the environment variables set. I tried restarting multiple times to no success. What I ended up doing was manually turning off MSI for the Function, then re-enabling it. This wasn't ideal, but it worked.
Hope it helps!
I've found out that if you enable MSI and then swap out the slot, the functionality leaves with the slot change. You can re-enable it by switching it off and on again but that will create a new identity in AD and will require you to reset permissions on the key vault for it to work.
Enable the identity and give access to your azure function app in keyvault via access policy.
You can find identity in platform feature tab
These two steps works for me
In my case I had forgotten to add an Access Policy for the application in the Key Vault
Just switched ON the Status like #Sebastian Inones showed.
Than add access policy for KeyVault like
This is resolved the issue!!
For the ones, like my self, wondering how to enable MSI.
My scenario:
I have an App Service already deployed and running for a long time.
In addition, on Azure DevOps I have my Pipeline configured to Auto-Swap my Deployment Slots (Staging/Production). Suddenly, after a normal push, Production starts failing because of the described issue.
So, in order to enable MSI again (I don't know why it has to be re-enabled but I believe this is only a workaround, not a solution, as it should be still enabled in the first place)
Go to your App Service. Then Under Settings --> Identity.
Check the status: In my case, it was off
I have attached an image below to make it easier to follow.
For the folks that will come across these answers, I would like to share my experience.
I got this problem with Azure Synapse pipeline run. Essentially I added access policies properly to the KeyVault, and also I added a LinkedService to the Azure Synapse pointing to my KeyVault.
If I trigger the notebook manually it works, but in the pipeline, it fails.
Initially, I used the following statement:
url = TokenLibrary.getSecret("mykeyvault", "ConnectionString")
Then I added the name of the linked service as a third parameter, and the pipeline was able to leverage that linked service to obtain the MSI token for a Vault.
url = TokenLibrary.getSecret("mykeyvault", "ConnectionString", "AzureKeyVaultLinkedServiceName")
Might be unrelated to your issue but I was getting the same error message.
For me, the issue was using pip3's azure-cli. I was able to fix this issue by using brew packages for both azure-cli and azure-functions-core-tools.
Uninstall pip3 azure-cli
pip3 uninstall azure-cli
Install brew azure-cli
brew update
brew install azure-cli
Double check if the error message ends with:
Please go to Tools->Options->Azure Services Authentication, and re-authenticate the account you want to use.