How to securely login in Az CLI from a DevOps Pipeline - azure

I want to execute AZ cli commands from my Azure DevOps Pipeline. In my YAML file I have this:
trigger:
- master
pool:
vmImage: 'ubuntu-latest'
variables:
buildConfiguration: 'Release'
steps:
- task: UsePythonVersion#0
inputs:
versionSpec: '3.x'
architecture: 'x64'
# Updating pip to latest
- script: python -m pip install --upgrade pip
displayName: 'Upgrade pip'
# Updating to latest Azure CLI version.
- script: pip install --pre azure-cli --extra-index-url https://azurecliprod.blob.core.windows.net/edge
displayName: 'upgrade azure cli'
- script: az --version
displayName: 'Show Azure CLI version'
- script: az extension add -n azure-devops
displayName: 'Install Azure DevOps Extension'
- script: echo ${AZURE_DEVOPS_CLI_PAT} | az devops login
env:
AZURE_DEVOPS_CLI_PAT: $(System.AccessToken)
displayName: 'Login Azure DevOps Extension'
- script: az aks show --name census-k8s --resource-group Census
displayName: 'Show AKS'
The echo ${AZURE_DEVOPS_CLI_PAT} | az devops login step is completed (with success apparently) with a warning message
Failed to store PAT using keyring; falling back to file storage.
You can clear the stored credential by running az devops logout.
Refer https://aka.ms/azure-devops-cli-auth to know more on sign in with PAT.
The az aks show step fails:
Please run 'az login' to setup account.
I am a little bit lost. The az devops login command should enable me to use the az cli, right? If not, Am I supposed to use az login instead of az devops login? And if I am supposed to use az login, how can I pass my credentials in a secure way?

No, you don't need az devops login. What you need is Azure CLI Task:
- task: AzureCLI#2
displayName: Azure CLI
inputs:
azureSubscription: <Name of the Azure Resource Manager service connection>
scriptType: ps
scriptLocation: inlineScript
inlineScript: |
az --version
az account show
but then you don't have to do any login. Please call there your az aks show --name census-k8s --resource-group Census

Just to Add to Krzysztof's answer (and jeromerg question in the comment): in Azure CLI step you can also use other tools then az, which require being logged in with AzureCLI:
- task: AzureCLI#2
displayName: Publish Function
inputs:
azureSubscription: <Name of the Azure Resource Manager service connection>
scriptType: ps
scriptLocation: inlineScript
inlineScript: |
func azure publish <function-name>

If your scriptLocation is a scriptPath use the following example
- task: AzureCLI#2
displayName: 'update function appsettings'
inputs:
azureSubscription: 'MY-AzureSubscriptionName'
scriptType: ps
scriptLocation: 'scriptPath'
scriptPath: '$(System.DefaultWorkingDirectory)/Scripts/updateSettings.ps1'
arguments:
-ResourceGroupName 'MY-ResourceGroupName' `
-FunctionAppName 'MY-FunctionAppName'
updateSettings.ps1
param (
[string]$ResourceGroupName,
[string]$FunctionAppName)
)
.
. script body here
.

To use Azure CLI from a script (powershell or batch) you must assign $(System.AccessToken) to an environment variable named AZURE_DEVOPS_EXT_PAT.
- pwsh: |
az pipelines build list
displayName: 'Show build list'
env:
AZURE_DEVOPS_EXT_PAT: $(System.AccessToken)
Source: https://learn.microsoft.com/en-us/azure/devops/cli/azure-devops-cli-in-yaml?view=azure-devops

Related

Change ARR affinity property from General Settings with Azure Devops pipeline

I'm trying to modify the "ARR affinity" (clientAffinityEnabled) property in App Service General Settings with a pipeline task but it doesn't work, the value doesn't change.
This pipeline works OK with other General Settings properties.
Another approach to solve this?
Azure DevOps pipeline task:
- task: AzureAppServiceSettings#1
inputs:
azureSubscription: XXXXXXX
ResourceGroupName: XXXXXXX
appName: XXXXXXX
generalSettings: |
[
{
"clientAffinityEnabled": false
}
]
Test the same settings in the AzureAppServiceSettings task, I can reproduce the same situation. It seems that the AzureAppServiceSettings task is not able to update the ARR affinity value.
For a workaround, you can change to use Azure CLI Task to run the Azure CLI: az webapp update to update the ARR affinity value.
For example:
steps:
- task: AzureCLI#2
displayName: 'Azure CLI '
inputs:
azureSubscription: xx
scriptType: ps
scriptLocation: inlineScript
inlineScript: 'az webapp update --name xx --resource-group xx --client-affinity-enabled false'

helm registry login --password-stdin in Azure DevOps pipeline

I am trying to login to my private ACR using azure DevOps pipeline.
I tried it this way:
- task: AzureCLI#2
inputs:
azureSubscription: $(azureSubscriptionForACR)
scriptType: 'ps'
scriptLocation: 'inlineScript'
inlineScript: |
$password = az acr credential show -n $(azureAcrName) --query passwords[0].value
helm registry login $(azureContainerRegistry) --username $(azureAcrUserName) --password $password
which works, but there is a warning when I run the pipeline:
"WARNING: Using --password via the CLI is insecure. Use --password-stdin."
I would like to avoid the warning, so I tried many variant of this, but no success:
- task: AzureCLI#2
inputs:
azureSubscription: $(azureSubscriptionForACR)
scriptType: 'ps'
scriptLocation: 'inlineScript'
inlineScript: |
$password = az acr credential show -n $(azureAcrName) --query passwords[0].value
echo $password | helm registry login $(azureContainerRegistry) --username $(azureAcrName) --password-stdin
It always end up with:
Error: Get "https://azureacr.azurecr.io/v2/": unauthorized: authentication required, visit https://aka.ms/acr/authorization for more information.
I am using new helm 3.8.0
Is there a way to do it with --password-stdin?
You can store the $password value as an Environment Variable in Azure Devops , the same way you are doing for the ACR username and other values and then use echo command .
Example:
First get the password for the ACR using the below command and then store it in Environment Variable registryPassword .
az acr credential show -n $(azureAcrName) --query passwords[0].value
Then use the below to login:
- task: AzureCLI#2
inputs:
azureSubscription: $(azureSubscriptionForACR)
scriptType: 'ps'
scriptLocation: 'inlineScript'
inlineScript: |
echo $(registryPassword) | helm registry login $(azureContainerRegistry) --username $(azureAcrName) --password-stdin
For more information you can refer this Blog by Abhith Rajan or
this SO thread.

Deploy/update Azure ML Model Using Az Ml CLI in Azure pipelines

I'm trying to deploy the azure ml model if not exists in the workspace and when the model is already available in the registered workspace then update the model with the latest version only when an update is available, but I don't know how this works in practice.
The Azure pipelines will run on a weekly schedule.
steps:
- task: AzureCLI#2
displayName: 'Install AML CLI'
inputs:
azureSubscription: $(ml_ws_connection)
scriptLocation: inlineScript
scriptType: 'bash'
inlineScript: 'az extension add -n azure-cli-ml'
- task: AzureCLI#2
displayName: 'Attach folder to workspace'
inputs:
azureSubscription: $(ml_ws_connection)
workingDirectory: $(ml_path)
scriptLocation: inlineScript
scriptType: 'bash'
inlineScript: 'az ml folder attach -w $(ml_ws) -g $(ml_rg)'
# Add potential automated tests
- task: AzureCLI#2
displayName: 'Create AKS cluster'
inputs:
azureSubscription: $(ml_ws_connection)
workingDirectory: $(ml_path)
scriptLocation: inlineScript
scriptType: 'bash'
inlineScript: 'az ml computetarget create aks --name $(ml_aks_name) --cluster-purpose DevTest'
- task: AzureCLI#2
displayName: 'Deploy model to AKS '
inputs:
azureSubscription: $(ml_ws_connection)
workingDirectory: $(ml_path)
scriptLocation: inlineScript
scriptType: 'bash'
inlineScript: 'az ml model deploy --name model1_aks --ct $(ml_aks_name) --ic config/inferenceConfig.json -e $(ml_env_name) --ev $(ml_env_version) --dc config/aksDeploymentConfig-aks.json --overwrite'
- task: AzureCLI#2
displayName: 'Update model in AKS '
inputs:
azureSubscription: $(ml_ws_connection)
workingDirectory: $(ml_path)
scriptLocation: inlineScript
scriptType: 'bash'
inlineScript: 'az ml service update --name $(deploy_service_name)
We can do CICD for automated model release as documented here Continuously deploy Azure Machine Learning models - Azure Machine Learning | Microsoft Docs.
Please follow the references for CLI spec.
Here is link to update a deployment using configuration.
The ml extension to the Azure CLI is the improved interface for Azure Machine Learning users. It enables you to train and deploy models from the command line, with features that accelerate scaling the data science process up and out, all while tracking the model lifecycle.
Using the CLI enables you to run distributed training jobs on GPU compute, automatically sweep hyperparameters to improve your results, and then monitor jobs in the AML studio user interface to see all details including important metrics, metadata and artifacts like the trained model, checkpoints and logs.
Additionally, the CLI is optimized to support YAML-based job, endpoint, and asset specifications to enable users to create, manage, and deploy models with proper CI/CD (or GitOps) best practices for an end-to-end MLOps solution.
To get started with the 2.0 machine learning CLI extension for Azure, please check the link here.

Deploy packages to multiple webapp service by azure pipeline single stage

I have more than 100 webapp service in azure. I want to deploy packages in 100 webapps by azure pipeline with one pipeline yml file. But I couldn't find any documentation like this. I got one microsoft documentation and they prefer to increase pipeline steps. If I have 100 webapps service, then have to add 100 steps for each deployment. This is not an efficient way and its time consuming. I want just like this step.
- task: AzureWebApp#1
displayName: 'Azure Web App Deploy'
inputs:
azureSubscription: '$(Parameters.connectedServiceName)'
appType: webApp
ResourceGroupName: $(group)
appName: 'JustGoTestAgain, justgotesttwo, webapp123, webapp555, webapp777 and so on ........'
package: '$(build.artifactstagingdirectory)/**/*.zip'
This yaml file is showing error. I couldn't find any essential extensions to fix it. I also couldn't find any azure powershell deployment command regarding to this issue. How can I get the solution?
You will not be able to do this like this. However you can use Azure Cli task:
- task: AzureCLI#2
displayName: Azure CLI
inputs:
azureSubscription: '$(Parameters.connectedServiceName)'
scriptType: ps
scriptLocation: inlineScript
inlineScript: |
$apps= #('JustGoTestAgain, justgotesttwo, webapp123, webapp555, webapp777 and so on ........')
foreach ($app in $apps) {
az webapp deployment source config-zip -g $(group) -n $app --src '$(build.artifactstagingdirectory)/SOME_FOLDER/Artifact.zip'
}
And here you have more details about deployment itself
Annother approach with multiple task bu continuation if one fail is:
parameters:
- name: apps
type: object
default:
- JustGoTestAgain
- justgotesttwo
- and so on
steps:
- ${{ each app in parameters.apps}}:
- task: AzureWebApp#1
displayName: 'Azure Web App Deploy ${{ app }}'
continueOnError: true
inputs:
azureSubscription: '$(Parameters.connectedServiceName)'
appType: webApp
ResourceGroupName: $(group)
appName: ${{ app }}
package: '$(build.artifactstagingdirectory)/**/*.zip'
Thete was issue with space. Now is fine. Apart from that there is only one issue with connectedServiceName
Job Job: Step input azureSubscription references service connection $(Parameters.connectedServiceName) which could not be found. The service connection does not exist or has not been authorized for use. For authorization details, refer to https://aka.ms/yamlauthz. Job Job: Step input azureSubscription references service connection $(Parameters.connectedServiceName) which could not be found. The service connection does not exist or has not been authorized for use. For authorization details, refer to https://aka.ms/yamlauthz. Job Job: Step input azureSubscription references service connection $(Parameters.connectedServiceName) which could not be found. The service connection does not exist or has not been authorized for use. For authorization details, refer to https://aka.ms/yamlauthz.
Which I skipped here as you already have it on your solution.

how to execute azure cli command in azure devops pipeline

To execute ACR commands in Azure CLI, we need upgrade AZ CLI. Need to have complete steps in YML to execute the AZ commands in pipeline.
Please refer link: https://learn.microsoft.com/en-us/azure/devops/cli/azure-devops-cli-in-yaml?view=azure-devops
For Linux: azure-pipelines-steps-linux.yml:
steps:
Updating to latest Azure CLI version.
script: pip install --pre azure-cli --extra-index-url https://azurecliprod.blob.core.windows.net/edge
displayName: 'upgrade azure cli'
script: az --version
displayName: 'Show Azure CLI version'
script: az extension add -n azure-devops
displayName: 'Install Azure DevOps Extension'
task: AzureCLI#2
inputs:
azureSubscription: 'AzureServiceConnection'
scriptType: bash
scriptLocation: 'inlineScript'
inlineScript: |
az account show

Resources