I am trying to use parameters in Azure Devops templates.
I can print any parameter inside the template.
But when I use parameter in a template with any task that requires azure subscription that will make the pipeline always fail with
"The pipeline is not valid. Job myDeployment: Step input
azureSubscription references service connection $(mySubscription)
which could not be found."
Example of pipeline and template below.
Is there any way to path azure Subscription to the template?strong text
pipeline.yml
- stage: myStage
pool: windows
variables:
- name: azureSubscription
value: mySubscription
- name: keyVaultName
name: myKeyVauld
jobs:
deployment: myDeployment
strategy:
runOnce:
deploy:
steps:
- template: myTemplate.yml
parameters:
subscription: $(azureSubscription) # changing this to literal will work but not what I need
vault: $(keyVaultName)
myTemplate.yml
parameters:
- name: subscription
type: string
default: ''
- name: vault
type: string
default: ''
steps:
- task: AzureKeyVault#1
inputs:
azureSubscription: '${{ parameters.subscription }}'
keyVaultName: '${{ parameters.vault }}'
secretsFilter: myKey
This is a known issue / limitation. You have to pass the Azure subscription as a literal. No way around it that I know of, unfortunately.
It's been a point of discussion for literally years on this GitHub issue: https://github.com/microsoft/azure-pipelines-agent/issues/1307
You need to go to Project setting -> Pipelines section -> Service connections and create a Service Connection for Azure Resource Manager, choose between Service principal and Managed identity authentication type.
After you can use the name of created Service Connection in your YAML file in azureSubscription parameter.
Related
I have a pipeline that is supposed to create a Azure subscription, to provide values I have set up a variable calld serviceConnection:
jobs:
- job: Variables
displayName: 'Variables'
steps:
- checkout: self
- task: Bash#3
displayName: 'Get Json values and set output variables'
name: setOutput
inputs:
targetType: inline
script: |
serviceConnection=$(cat $(System.DefaultWorkingDirectory)/test/*.json | jq -r '.test.serviceConnection')
echo "##vso[task.setvariable variable=serviceConnection;isoutput=true]$serviceConnection"
- job: CreateSubscription
displayName: 'Create Subscription to Azure'
dependsOn: Variables
variables:
- name: serviceConnection
value: $[ dependencies.Variables.outputs['setOutput.serviceConnection'] ]
- task: AzureCLI#2
name: CreateSubscriptionCli
displayName: 'Create a subscription'
inputs:
azureSubscription: $(serviceConnection)
But when i start the pipeline i get this error:
"The pipeline is not valid. Job CreateSubscription: Step CreateSubscription input connectedServiceNameARM references service connection $(serviceConnection) which could not be found.
when i print out the variable it shows the value as intended.
using syntax:
${{ parameters.serviceConnection}}
i get similar error but with
$[ dependencies.Variables.outputs['setOutput.serviceConnection'] ]
I cannot say that I understand at all what you are trying to do...but: The input "azureSubscription" for the AzureCLI#2 task, is a well-defined input which needs to reference an existing Azure DevOps Service Connection.
"The pipeline is not valid. Job CreateSubscription: Step CreateSubscription input connectedServiceNameARM references service connection $(serviceConnection) which could not be found.
From the YAML sample, you are using the cross-jobs variable to pass the service connection value.
The variable value will be set at runtime. But the Azure CLI task will read the azureSubscription Service connection value at compile time.
In this case, the variable $(serviceConnection) in Azure CLI task will not able to get value at compile time and show the error above.
For more detailed info, you can refer to this doc about cross-jobs variables.
For a workaround, you can split the pipeline to two. One is used to get the service connection value in Json file. And then the first pipeline can trigger the second pipeline and pass the value to the second pipeline to run Azure CLI task.
I want to deploy more than 100 webapps in one azure pipeline. I want to put these webapp name in azure key vault. But there has only one value can be stored in one variable. Is there any way to put multiple value or object in one variable in azure key vault.
parameters:
- name: apps
type: object
default:
- JustGoTestAgain
- justgotesttwo
- and so on
steps:
- ${{ each app in parameters.apps}}:
- task: AzureWebApp#1
displayName: 'Azure Web App Deploy ${{ app }}'
continueOnError: true
inputs:
azureSubscription: '$(Parameters.connectedServiceName)'
appType: webApp
ResourceGroupName: $(group)
appName: ${{ app }}
package: '$(build.artifactstagingdirectory)/**/*.zip'
I have also put multiple value using comma in azure secret vault. Is this the right way to store multiple value? I ran pipeline but it completes with this warning
##[error]Error: Failed to fetch App Service '' publishing credentials. Error: The Resource 'Microsoft.Web/sites/' under
resource group '***' was not found. For more details please go to
https://aka.ms/ARMResourceNotFoundFix (CODE: 404)
How to solve this problem?
If you really need to store a bunch of variables in a single object in Key Vault, I recommend that you use a JSON object with a bunch of key/value pairs. That way you can deserialize it quickly.
I am looking at deploying a .Net Core WebApi service to an Azure App Service and as part of the deployment I am keen to update the connection string in the appsettings.json with the CosmosDb connection string. I have a Azure KeyVault which has the connection string stored in there as a secret.
Using the YAML build pipeline for CI/CD I have the following (snippet) from my pipeline
- task: AzureKeyVault#1
inputs:
azureSubscription: '<service-principal>'
KeyVaultName: '<keyvault-name>'
SecretsFilter: '*'
RunAsPreJob: true
- task: AzureRmWebAppDeployment#4
inputs:
ConnectionType: 'AzureRM'
azureSubscription: '<service-principal>'
appType: 'webApp'
WebAppName: '<ci-resource-group>'
VirtualApplication: '/'
packageForLinux: '$(System.DefaultWorkingDirectory)/**/*.zip'
JSONFiles: '**/appsettings.json'
These two tasks are in a stage which starts with downloading the published artifact from a previous stage.
So the Azure App Service Deploy task can do JSON transformation but I need to define a variable in the format ConnectionStrings:CosmosDb with the value from the secret stored in the keyvault and that I am not certain of how to do!
Firstly, is this the correct way? I have seen articles about using a reference to the secret in the keyvault, is that the correct way?
The keyvault secrets are available to the pipeline using $(secret), how can I create a variable for the AzureRmWebAppDeployment#4 task as above?
Everything I have found so far points to the Classic release pipelines, and using variable groups but this needs to be part of the YAML pipeline.
Following the docs
To substitute values in nested levels of the file, concatenate the names with a period (.) in hierarchical order.
But your secrets cannot have dots in name so you must do a small rewrite between those two steps:
- powershell: |
echo "##vso[task.setvariable variable=ConnectionStrings.CosmosDb;]$(ConnectionStrings-CosmosDb)"
where your secret is ConnectionStrings-CosmosDb
I'm using Yaml for Azure devops pipeline.
I am using a hierarchical style for that.
I'm having one top level yaml: feature.yaml
which has following structure:
trigger:
...
pool:
vmImage: ...
variables:
group: ...
stages:
- template: deploy.yaml
parameters:
subName: $(subscription) #This should be taken from Variable group
I have deploy.yaml as:
stages:
- stage: deploy
jobs:
- job: deploy
steps:
- task: AzureKeyVault#1
inputs:
azureSubscription: $(paramaters.subName) #This should be resolved from parameter passed form feature.yaml
KeyVaultName: ...
SecretsFilter: '*'
RunAsPreJob: true
However, whenever I run this from Azure DevOps, I'm getting this error:
There was a resource authorization issue: "The pipeline is not valid. Job deploy: Step AzureKeyVault input ConnectedServiceName references service connection $(paramaters.subName) which could not be found. The service connection does not exist or has not been authorized for use. For authorization details, refer to https://aka.ms/yamlauthz."
It seems pipeline is not able to resolve value of azureSubscription name from variable group.
Any suggestions?
I discovered that when the YAML is initially parsed it expects the variable group to be in scope. I needed to move my variable group to the top of the YAML file and then it found the azure subscription variable. Not what I was expecting.
I think you have to define the parameters in the deploy.yaml!
https://learn.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=azure-devops&tabs=example%2Cparameter-schema
for those who are looking for resolution,
Here's what you'll need to do:
In the Top-level feature.yaml, Add the desired variable group name and pass in the parameter from the template which has the azure subscription name:
variables:
- group: xxx
- template: deploy.yaml
parameters:
subName: $(_subscriptionName)
And utilize that parameter in template deploy.yaml:
- task: AzureKeyVault#1
inputs:
azureSubscription: ${{ parameters.subName}}
keyVaultName: xxx
secretsFilter: '*'
Its a known issue currently that we can not use variable from variable group directly to azureSubscription input of AzureKeyVault task as per this thread.
Thi work around should work fine!
We had a similar issue to this and it was because we missed off the - in the yaml under variables:
variables:
group: ...
To:
variables:
- group: ...
This fixed the issue for us.
An Azure DevOps YAML based pipeline has been created from scratch and requires the use of Azure Subscription. The Subscription value has been stored within Key Vault and then linked to the Variable Group. Pipeline has unobtrusive access to both Variable Group and linked Key Vault. However the pipeline execution fails with the error that the pipeline does not have access to the required subscription. When the Subscription value is moved to the Variable Group the issue is still present. When the Subscription value is declared as a pipeline variable the issue is gone. The click on the Authorize button next to the error does not help with the issue.
- stage: 'DeployDevelopment'
displayName: ''
dependsOn: Build
jobs:
- deployment: DeployDevelopment
pool:
vmImage: 'ubuntu-latest'
environment: Development
variables:
- group: Secrets
- group: Release
strategy:
runOnce:
deploy:
steps:
- task: AzureRmWebAppDeployment#4
displayName: ''
inputs:
azureSubscription: '$(ConnectedServiceName)'
appType: 'webAppLinux'
WebAppName: '$(DevEnvironemntWebAppName)'
packageForLinux: '$(Pipeline.Workspace)/app/s'
RuntimeStack: 'NODE|10-lts'
StartupCommand: '$(StartupCommand)'
WebConfigParameters: '-Handler iisnode -NodeStartFile server.js -appType node'
AppSettings: '-WEBSITE_NODE_DEFAULT_VERSION 10.12.0'
From Developer Community
Thanks for reporting the issue on Developer Community. Azure key vault
values are fetched at run time. The resource are authorized before
deployment. Pipeline can;t be authorized for a value that is not
available. Hence this is not supported. scenario.