When working with AWS, if you use aws configure to log in, you can use the AWS SDK without exposing credentials in any programming language from your local machine. If anything is running inside aws later (Lambda, EC2, whatever) the exact same code does use the resource assigned IAM Role without any configuration.
I try to get the same to work with Azure, I thought that the Azure.Identity.DefaultAzureCredential does do this. But I can't even run my code locally:
var blobServiceClient = new BlobServiceClient(storageUri, new DefaultAzureCredential());
var containerClient = await blobServiceClient.CreateBlobContainerAsync("test-container");
How can I get a BlobServiceClient that authenticates using the CLI creds on my local machine, and a managed identity if running inside an AppService.
In your scenario, as you used, the DefaultAzureCredential is the best choice along with the BlobServiceClient, but it does not use CLI credentials to authenticate.
To make it work, just set the Environment variables with AZURE_CLIENT_ID, AZURE_TENANT_ID, AZURE_CLIENT_SECRET of your service principal. In Azure, it uses the MSI to authenticate.
If you want to use CLI credentials to authenticate, there is AzureServiceTokenProvider, it can also access azure storage, but you could not use it along with BlobServiceClient, you need to get the access token with the resource https://storage.azure.com,
var azureServiceTokenProvider2 = new AzureServiceTokenProvider();
string accessToken = await azureServiceTokenProvider2.GetAccessTokenAsync("https://storage.azure.com").ConfigureAwait(false);
then use the access token to call Storge REST API, I think the first option is more convenient, to use which one, it is up to you.
Related
Usually in Python what I do, I get the application default credentials, I get the access token then I refresh it to be able to authenticate to a private environment.
Code in Python:
# getting the credentials and project details for gcp project
credentials, your_project_id = google.auth.default(scopes=["https://www.googleapis.com/auth/cloud-platform"])
#getting request object
auth_req = google.auth.transport.requests.Request();
print(f"Checking Authentication : {credentials.valid}")
print('Refreshing token ....')
credentials.refresh(auth_req)
#check for valid credentials
print(f"Checking Authentication : {credentials.valid}")
access_token = credentials.token
credentials = google.oauth2.credentials.Credentials(access_token);
storage_client = storage.Client(project='itg-ri-consumerloop-gbl-ww-dv',credentials=credentials)
I am entirely new to NodeJS, and I am trying to make the same thing.
My goal later is to create an app engine application that would expose an image that is found in a private bucket, so credentials are a must.
How it is done?
For authentication, you could rely on the default application credentials that are present within the GCP platform (GAE, Cloud Functions, VM, etc.). Then you could just run the following piece of code from the documentation:
const {Storage} = require('#google-cloud/storage');
const storage = new Storage();
const bucket = storage.bucket('albums');
const file = bucket.file('my-existing-file.png');
In most circumstances, there is no need to explicitly use authentication packages since they are already executed underneath the google-cloud/storage package in Nodejs. The same holds for the google-cloud-storage package in Python. It could help to look at the source code of both packages on Github. For me, this really helped to understand the authentication mechanism.
When I develop code on my own laptop, that interacts with google cloud storage, I first tell the gcloud SDK what my credentials are and on which GCP project I am working. I use the following commands for this:
gcloud config set project [PROJECT_ID]
gcloud auth application-default login
You could also set DEFAULT_APPLICATION_CREDENTIALS as an environment variable that points to a credentials file. Then within your code, you could pass the project name when initializing the client. This could be helpful if you are running your code outside of GCP on another server for example.
I need to read the list of users in the Azure active directory. The client has created a Graph API application but they do not want to share the client secret of the application, instead they asked us to use the Key vault. How to access from the node.js application the key to retrieve the list of users?
I tried the below one but gave error and I am not sure how to authenticate.
const { DefaultAzureCredential } = require("#azure/identity");
const { SecretClient } = require("#azure/keyvault-secrets");
const credential = new DefaultAzureCredential();
const vaultName = "lsm-keyvault";
const url = `https://${vaultName}.vault.azure.net`;
const client = new SecretClient(url, credential);
const secretName = "Demo";
async function main() {
const result = await client.setSecret(secretName, "MySecretValue", {
enabled: false
});
console.log(result)
}
Well, if you run the code in local, the DefaultAzureCredential will use the environmental variables automatically.
So in your case, you need to register an application with Azure AD, and get the tenant id, client id(i.e. application id), client secret(i.e. application secret), set the environmental variables, AZURE_CLIENT_ID, AZURE_CLIENT_SECRET, and AZURE_TENANT_ID.
For the 403 error you got, I notice you said It added as a compound entity, based on my experience, you did not add the correct service principal related to the AD App correctly to the Access policies of the keyvault. If you add it correctly, it will appear as APPLICATION, not COMPOUND IDENTITY.
So when you add it, you could search for the client Id(i.e. application Id) or the name of your App Registration directly, make sure you add the correct one. I gave the details in this similar issue, you could refer to it.
To retrieve the secret, the Get permission is enough, the code should be
const retrievedSecret = await client.getSecret(secretName);
I notice you use client.setSecret in your code, it is used to save a secret, to use it, you may need the Set permission.
For more details, see Quickstart: Azure Key Vault client library for Node.js (v4).
Update:
I have to eventually need to deploy this but not in azure but in another environment. How do I set the environment variables and access it.
If so, you need to change your code to authenticate, use the three values directly in the code.
Change the lines
const { DefaultAzureCredential } = require("#azure/identity");
const credential = new DefaultAzureCredential();
To
const { ClientSecretCredential } = require("#azure/identity");
const credential = new ClientSecretCredential(tenantId, clientId, clientSecret);
See - https://www.npmjs.com/package/#azure/identity/v/1.0.3#authenticating-as-a-service-principal
All you need to do is follow the below steps:
Create an App in the Azure Active Directory (Service Principal) from App Registrations.
Go to Key Vault resource, Access Policy blade, assign read access to this Azure AD App (Service Principal) that we created in the above step.
Set these 3 Environment variables AZURE_CLIENT_ID, AZURE_TENANT_ID, and AZURE_CLIENT_SECRET in your App Service. Get the values of these variables from the app that we created in step 1.
Use DefaultAzureCredential that we are already using now. This will automatically pick the credentials from the environment variables that we defined in App Service for the authentication.
Another way is to obtain Key Vault token dynamically and use that token to get the secrets from the Key Vault - https://learn.microsoft.com/en-us/samples/azure-samples/app-service-msi-keyvault-node/app-service-msi-keyvault-node/
Helpful Reference:
https://www.rahulpnath.com/blog/defaultazurecredential_from_azure_sdk/
I'm working with Azure Key Vault and I'm testing the "Managed Identities" for Azure Resource. Long story short: with this feature we can easily access to a KeyVault secrets from (e.g.) VM, App Service... I wrote this simple code:
private void GetKeyVaultSecrets()
{
AzureServiceTokenProvider azureServiceTokenProvider = new AzureServiceTokenProvider();
var akvCallback = new KeyVaultClient.AuthenticationCallback(azureServiceTokenProvider.KeyVaultTokenCallback);
var keyVaultClient = new KeyVaultClient(akvCallback);
var secret = keyVaultClient.GetSecretAsync("https://mykeyvault.vault.azure.net/secrets/mySecret").GetAwaiter().GetResult();
string superSecret = secret.Value;
}
And I'm able to access my "Secret" without any kind of Authorization. Is it safe? Am I missing something? It looks so strange that with those 7 lines of code you have access to all your secrets. Thanks for your attention.
If you are running this on a service with a system-assigned Managed Identity, here's what actually happens (example for App Service, VM is slightly different):
Your app reads IDENTITY_ENDPOINT and IDENTITY_HEADER environment variables
HTTP call to IDENTITY_ENDPOINT using the IDENTITY_HEADER as authentication
This endpoint cannot be called from the outside, only from within the instance.
Its port is also random
In the call, your app specifies it wants a token for Key Vault (resource https://vault.azure.net)
The Managed Identity endpoint uses the certificate it has created to authenticate to Azure Active Directory with the Client Credentials flow
Azure AD verifies the request and issues a token
Managed Identity endpoint returns the token to your app
KeyVaultClient uses the token to authorize the call to Key Vault
On Virtual Machines, the Instance Metadata Service is used to get tokens.
The main thing to understand is that any process running on the instance itself is capable of getting tokens from the Managed Identity.
Though if you were to get malicious code running on your instance,
other approaches could be in trouble as well :)
If you run that code locally, it can work as well.
AzureServiceTokenProvider also attempts Visual Studio authentication as well as Azure CLI authentication.
So if you are logged in to e.g. AZ CLI, it is able to get an access token for your user to Azure Key Vault and access it.
Reference: https://learn.microsoft.com/en-us/azure/app-service/overview-managed-identity?context=azure%2Factive-directory%2Fmanaged-identities-azure-resources%2Fcontext%2Fmsi-context&tabs=dotnet#rest-protocol-examples
Example request done in VMs: https://learn.microsoft.com/en-us/azure/active-directory/managed-identities-azure-resources/tutorial-windows-vm-access-nonaad#access-data
I was using azure-keyvault but it's now deprecated. I'm using for encrypting and decrypting stuff. All I needed to operate was clientId, clientSecret and the key identifier.
Because now azure-keyvault is deprecated I switched to #azure/keyvault-keys/#azure/identity. For this I need a tenant (ClientSecretCredential) which I previously didn't need. Why is that or is there a way to not needing it?
When using a ClientSecretCredential, because a service principal belongs to a particular tenant, you have to specify that tenant rather than it coming back in a callback like the older code.
There are many different credential types, but our recommendation is to use DefaultAzureCredential which supports MSI, environment credentials (service principal using $AZURE_TENANT_ID, $AZURE_CLIENT_ID, and $AZURE_CLIENT_SECRET), and interactive browser login for most languages - soon with more credentials like azure CLI and Visual Studio. With support for azure CLI, that provides parity with the older packages like you used and then some. Just by using DefaultAzureCredential you get all that and it supports different environments by default, so you don't have to change your code to use different credentials for dev, staging, or production environments.
So like in the referenced example, you just instantiate a DefaultAzureCredenial and thats it. If you have your service principal environment variables defined, they will be used if Managed Identity (MSI) wasn't detected.
import { SecretClient } from '#azure/keyvault-secrets';
import { DefaultAzureCredential } from '#azure/identity';
import { CosmosClient } from '#azure/cosmos';
const keyVaultUrl = process.env('APP_KEY_VAULT_URI');
const credential = new DefaultAzureCredential();
let storageClient;
let cosmosClient;
async function configureClients() {
const kvClient = new SecretClient(keyVaultUrl, credential);
const storageUri = await client.getSecret('storageUri');
const cosmosDbConnectionString = await client.getSecret('cosmosDb');
cosmosClient = new CosmosClient(cosmosDbConnectonString);
storageClient = new BlobServiceClient(storageUri, credential);
The order of credentials is optimized for production workloads, but supports developer machines - pretty close to the order I listed them above.
I am trying to access Azure Key Vault from my Azure App Service. I followed the steps outlined on this documentation: https://learn.microsoft.com/en-us/azure/key-vault/managed-identity (turned on system assigned identity for the app service, updated the access policy of the key vault to include the app with Get,List secret permissions).
However, when I run the web application, it is not able to get the secret from the key vault and my web service hits the following error:
502 - Web server received an invalid response while acting as a gateway or proxy server.
There is a problem with the page you are looking for, and it cannot be displayed. When the Web server (while acting as a gateway or proxy) contacted the upstream content server, it received an invalid response from the content server.
This is what my code looks like:
AzureServiceTokenProvider azureServiceTokenProvider = new AzureServiceTokenProvider();
KeyVaultClient keyVaultClient = new KeyVaultClient(new KeyVaultClient.AuthenticationCallback(azureServiceTokenProvider.KeyVaultTokenCallback));
var secret = keyVaultClient.GetSecretAsync(KeyVaultUrl);
authenticationKey = secret.Result.Value;
The service gets stuck on the secret.Result.Value line. Is there something else I need to do?
This is much easier with the new package, like Azure.Security.KeyVault.Secrets. Together with Azure.Identity, you can just pass a DefaultAzureCredential like in our samples.
var client = new SecretClient(
new Uri("https://myvault.vault.azure.net"),
new DefaultAzureCredential());
KeyVaultSecret secret = await client.GetSecretAsync("secret-name");
string authenticationKey = secret.Value;
The DefaultAzureCredential is optimized to work for managed identity, service principals from the environment, and interactive logins to support the same code running both in production and on developer machines. The new libraries are also faster with fewer allocations, and have much better diagnostics - defaulted to on when using Azure Application Monitor.
They target netstandard2.0 so should be compatible with the older packages these replace. Would you be able to upgrade? We're only making critical fixes for the older packages, and recommending people upgrade to Azure.* packages intead of the older Microosft.Azure.* packages.
As for the problem itself, it's hard to say without knowing when you're calling this in your application. During startup? What version of .NET? What are you using for your ASP.NET application framework?
While it's probably not the cause of the problem, it's hard to ignore that you're calling an async method synchronously, which can also cause problems. If you're in an async method, you should write your code like so:
var secret = await keyVaultClient.GetSecretAsync(KeyVaultUrl);
authenticationKey = secret.Value;
If not, call:
var secret = keyVaultClient.GetSecretAsnc(KeyVaultUrl).GetAwaiter().GetResult();
This is not recommended, though. In the new packages I mentioned above, we have both sync and async versions that are either sync or async all the way through the call stack and safer to use. Generally, though, you should use async calls - especially for network traffic like accessing Key Vault because, depending on what thread you call it, it can hang your UI.
When you test in local:
Add your vs signed account into azure keyvault. Go to keyvault> Access policy> add your account with get secret permmission.
When you publish to azure:
1.Enable webapp MSI.
2.Go to keyvault> Access policy> add your webapp's service principal with get secret permission.
The code:
AzureServiceTokenProvider azureServiceTokenProvider = new AzureServiceTokenProvider();
KeyVaultClient keyVaultClient = new KeyVaultClient(new Microsoft.Azure.KeyVault.KeyVaultClient.AuthenticationCallback(azureServiceTokenProvider.KeyVaultTokenCallback));
var secret = keyVaultClient.GetSecretAsync("https://yourkevaultname.vault.azure.net/secrets/secretname/437d301daxxxxxx");
var authenticationKey = secret.Result.Value;
ViewBag.Message = authenticationKey.ToString();