I want to deploy more than 100 webapps in one azure pipeline. I want to put these webapp name in azure key vault. But there has only one value can be stored in one variable. Is there any way to put multiple value or object in one variable in azure key vault.
parameters:
- name: apps
type: object
default:
- JustGoTestAgain
- justgotesttwo
- and so on
steps:
- ${{ each app in parameters.apps}}:
- task: AzureWebApp#1
displayName: 'Azure Web App Deploy ${{ app }}'
continueOnError: true
inputs:
azureSubscription: '$(Parameters.connectedServiceName)'
appType: webApp
ResourceGroupName: $(group)
appName: ${{ app }}
package: '$(build.artifactstagingdirectory)/**/*.zip'
I have also put multiple value using comma in azure secret vault. Is this the right way to store multiple value? I ran pipeline but it completes with this warning
##[error]Error: Failed to fetch App Service '' publishing credentials. Error: The Resource 'Microsoft.Web/sites/' under
resource group '***' was not found. For more details please go to
https://aka.ms/ARMResourceNotFoundFix (CODE: 404)
How to solve this problem?
If you really need to store a bunch of variables in a single object in Key Vault, I recommend that you use a JSON object with a bunch of key/value pairs. That way you can deserialize it quickly.
Related
I have been writing some terraform and using Azure Devops to deploy the pipeline. However if I use a variable $(serviceconnection) for the service connection it fails with the following error:
There was a resource authorization issue: "The pipeline is not valid. Job DeployDev: Step TerraformCLI1 input backendServiceArm references service connection $(serviceconnection) which could not be found. The service connection does not exist or has not been authorized for use.
I Have tried authorising it but no luck. Is there any workaround?
The task is a YAML task to use terraform as below :
- task: charleszipp.azure-pipelines-tasks-terraform.azure-pipelines-tasks-terraform-cli.TerraformCLI#0
displayName: 'Terraform Init'
inputs:
command: init
workingDirectory: $(Agent.BuildDirectory)/a/azuredirectory/terraform
backendType: azurerm
backendServiceArm: $(serviceconnection)
backendAzureRmResourceGroupName: $(ResourceGroupName)
backendAzureRmStorageAccountName: $(StorageAccountName)
backendAzureRmContainerName: $(ContainerName)
backendAzureRmKey: $(AzureRmKey)
You need to use a Template expression syntax for the service connection variable:
backendServiceArm: ${{ variables.serviceconnection }}
I imagine it's because the service connection needs to be known before the pipeline runs.
Sample use case. Using a variable file called variable.dev.yaml:
variables:
serviceconnection: my-dev-service-connection-name
...
You could then reference that in your pipeline:
jobs:
- job: myJob
...
variables:
- template: ./variable.dev.yaml
steps:
- task: AzureCLI#2
inputs:
azureSubscription: ${{ variables.serviceconnection }}
...
If you want to use runtime variable like $(serviceconnection), it is not supported now.
You can use ${{ variables.serviceconnection }} as Thomas recommended. But this practice means that you have to specify variables in advance(Before you run the pipeline).
For service connections, you can specify a value directly or use the ’compile-time variable‘ ${{xxx}}, which will expand and then populate the service connection section with values before running. In this usage of $(xxx), the service connection of the task cannot be obtained, because this is a runtime value.
The service connection needs to be specified before running. The changes (runtime changes) of the variables during the pipeline run will not be acquired by the service connection part of the subsequent task.
You are using a runtime variable.
But run time variables aren't supported for service connection OR azure subscription. The variable will get initialized at the run time.
https://github.com/microsoft/azure-pipelines-tasks/issues/10376#issuecomment-514477023
You can follow below method to use different service connection:
https://stackoverflow.com/a/57520153/6261890
But still need point that, parameters are expanded just before the pipeline runs, hardcode the specific service connection is unavoidable, this is by design.
Also clearly in this official document:
https://learn.microsoft.com/en-us/azure/devops/pipelines/library/service-endpoints?view=azure-devops&tabs=yaml#use-a-service-connection
I am using AzureAppConfigurationPush#1 task for deploying app configuration values in different environment. I am able to add key value pairs sucessfully but I am not finding ways to add Key vault reference.
-task: AzureAppConfigurationPush#1
inputs:
azureSubscription: '${{ parameters.azureResourceManager }}'
ConfigstoreName: '${{ parameters.appConfigName }}'
ConfigurationFile: '${{parameters.configFileName }}'
Separator: ','
Strict: false
I am passing config file name as parameters in which I am giving Json which I export from azure app configuration. When I export app configuration values, key value pair contains uri and its value like this:
"AppInsightsKey": "{"uri":"https://.vault.azure.net/secrets/AppInsightsKey"}"
But this doesn't go as Key vault reference using push task but considered as a string completely.
Can you help me understand hot to push key vault reference so that it will automatically create a reference if possible?
The push task takes content type as a parameter. If you want to create key-vault references, you'll need to specify the content type of a key vault reference for the settings being pushed.
Key Vault Reference content type: application/vnd.microsoft.appconfig.keyvaultref+json;charset=utf-8
It seems that that is a problem with Azure App Configuration Push extension.
You have to contact with that team to report the problem.
I am looking at deploying a .Net Core WebApi service to an Azure App Service and as part of the deployment I am keen to update the connection string in the appsettings.json with the CosmosDb connection string. I have a Azure KeyVault which has the connection string stored in there as a secret.
Using the YAML build pipeline for CI/CD I have the following (snippet) from my pipeline
- task: AzureKeyVault#1
inputs:
azureSubscription: '<service-principal>'
KeyVaultName: '<keyvault-name>'
SecretsFilter: '*'
RunAsPreJob: true
- task: AzureRmWebAppDeployment#4
inputs:
ConnectionType: 'AzureRM'
azureSubscription: '<service-principal>'
appType: 'webApp'
WebAppName: '<ci-resource-group>'
VirtualApplication: '/'
packageForLinux: '$(System.DefaultWorkingDirectory)/**/*.zip'
JSONFiles: '**/appsettings.json'
These two tasks are in a stage which starts with downloading the published artifact from a previous stage.
So the Azure App Service Deploy task can do JSON transformation but I need to define a variable in the format ConnectionStrings:CosmosDb with the value from the secret stored in the keyvault and that I am not certain of how to do!
Firstly, is this the correct way? I have seen articles about using a reference to the secret in the keyvault, is that the correct way?
The keyvault secrets are available to the pipeline using $(secret), how can I create a variable for the AzureRmWebAppDeployment#4 task as above?
Everything I have found so far points to the Classic release pipelines, and using variable groups but this needs to be part of the YAML pipeline.
Following the docs
To substitute values in nested levels of the file, concatenate the names with a period (.) in hierarchical order.
But your secrets cannot have dots in name so you must do a small rewrite between those two steps:
- powershell: |
echo "##vso[task.setvariable variable=ConnectionStrings.CosmosDb;]$(ConnectionStrings-CosmosDb)"
where your secret is ConnectionStrings-CosmosDb
I am trying to use parameters in Azure Devops templates.
I can print any parameter inside the template.
But when I use parameter in a template with any task that requires azure subscription that will make the pipeline always fail with
"The pipeline is not valid. Job myDeployment: Step input
azureSubscription references service connection $(mySubscription)
which could not be found."
Example of pipeline and template below.
Is there any way to path azure Subscription to the template?strong text
pipeline.yml
- stage: myStage
pool: windows
variables:
- name: azureSubscription
value: mySubscription
- name: keyVaultName
name: myKeyVauld
jobs:
deployment: myDeployment
strategy:
runOnce:
deploy:
steps:
- template: myTemplate.yml
parameters:
subscription: $(azureSubscription) # changing this to literal will work but not what I need
vault: $(keyVaultName)
myTemplate.yml
parameters:
- name: subscription
type: string
default: ''
- name: vault
type: string
default: ''
steps:
- task: AzureKeyVault#1
inputs:
azureSubscription: '${{ parameters.subscription }}'
keyVaultName: '${{ parameters.vault }}'
secretsFilter: myKey
This is a known issue / limitation. You have to pass the Azure subscription as a literal. No way around it that I know of, unfortunately.
It's been a point of discussion for literally years on this GitHub issue: https://github.com/microsoft/azure-pipelines-agent/issues/1307
You need to go to Project setting -> Pipelines section -> Service connections and create a Service Connection for Azure Resource Manager, choose between Service principal and Managed identity authentication type.
After you can use the name of created Service Connection in your YAML file in azureSubscription parameter.
An Azure DevOps YAML based pipeline has been created from scratch and requires the use of Azure Subscription. The Subscription value has been stored within Key Vault and then linked to the Variable Group. Pipeline has unobtrusive access to both Variable Group and linked Key Vault. However the pipeline execution fails with the error that the pipeline does not have access to the required subscription. When the Subscription value is moved to the Variable Group the issue is still present. When the Subscription value is declared as a pipeline variable the issue is gone. The click on the Authorize button next to the error does not help with the issue.
- stage: 'DeployDevelopment'
displayName: ''
dependsOn: Build
jobs:
- deployment: DeployDevelopment
pool:
vmImage: 'ubuntu-latest'
environment: Development
variables:
- group: Secrets
- group: Release
strategy:
runOnce:
deploy:
steps:
- task: AzureRmWebAppDeployment#4
displayName: ''
inputs:
azureSubscription: '$(ConnectedServiceName)'
appType: 'webAppLinux'
WebAppName: '$(DevEnvironemntWebAppName)'
packageForLinux: '$(Pipeline.Workspace)/app/s'
RuntimeStack: 'NODE|10-lts'
StartupCommand: '$(StartupCommand)'
WebConfigParameters: '-Handler iisnode -NodeStartFile server.js -appType node'
AppSettings: '-WEBSITE_NODE_DEFAULT_VERSION 10.12.0'
From Developer Community
Thanks for reporting the issue on Developer Community. Azure key vault
values are fetched at run time. The resource are authorized before
deployment. Pipeline can;t be authorized for a value that is not
available. Hence this is not supported. scenario.