Is there a way to set a default value for an array of objects in a yaml template (Azure Pipelines) - azure

I have a yaml template that contains parameters like below. In the array / list I want to be able to set the default value for one of the properties (the object has 3). The reason being is I have some if statements that check these properties and I want them to run regardless of whether or not the property is set in the yaml that uses the template (the third property was added later and I don't want to have to update every repo that uses this template).
Is this something that can be done in the parameter setup?
Notes: Using the below example if the property WebProject is not used I would like everything to go through the template as if it was set to false. I know there is some duplication where I check the property and copy files / publish the artifacts but at the moment I want to get it working and make it more efficient afterwards.
yaml template
parameters:
- name: ArtifactPublish
type: boolean
default: false
- name: Solution
type: string
default: '**/*.sln'
- name: Artifacts
type: object
default: []
jobs:
- job: Build
displayName: 'Build, Pack, and Publish'
pool:
vmImage: 'windows-latest'
variables:
solution: ${{ parameters.Solution }}
buildPlatform: 'Any CPU'
buildConfiguration: 'Release'
- task: VSBuild#1
displayName: 'Build Solution'
inputs:
solution: '$(solution)'
msbuildArgs: '/p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:PackageLocation="$(Build.ArtifactStagingDirectory)"'
platform: '$(buildPlatform)'
configuration: '$(buildConfiguration)'
- ${{ if eq(parameters.ArtifactPublish, true) }}:
- ${{ each artifact in parameters.Artifacts }}:
- ${{ if eq(artifact.WebProject, false) }}:
- task: CopyFiles#2
displayName: 'Copy .artifactignore: ${{ artifact.ArtifactPath }}'
inputs:
SourceFolder: '$(Build.SourcesDirectory)'
Contents: '.artifactignore'
TargetFolder: '$(Build.SourcesDirectory)/${{ artifact.ArtifactPath }}'
- task: PublishPipelineArtifact#1
displayName: 'Publish Artifact: ${{ artifact.ArtifactName }}'
inputs:
targetPath: '$(Build.SourcesDirectory)/${{ artifact.ArtifactPath }}'
artifactName: '${{ artifact.ArtifactName }}'
- ${{ if eq(artifact.WebProject, true) }}:
- task: CopyFiles#2
displayName: '${{ artifact.ArtifactPath }}*'
inputs:
SourceFolder: $(Build.ArtifactStagingDirectory)
Contents: '${{ artifact.ArtifactPath }}*'
TargetFolder: '$(Build.ArtifactStagingDirectory)/${{ artifact.ArtifactPath }}'
- task: PublishPipelineArtifact#1
displayName: 'Publish Artifact: ${{ artifact.ArtifactName }} (WEB)'
inputs:
targetPath: $(Build.ArtifactStagingDirectory)/${{ artifact.ArtifactPath }}
artifactName: '${{ artifact.ArtifactName }}'
Example yaml that uses the template:
trigger:
- release
- development
- master
- feature/*
- task/*
resources:
repositories:
- repository: templates
name: Project/RepoName
type: git
ref: refs/heads/release
jobs:
- template: Templates/example.yml#templates
parameters:
ArtifactPublish: true
Artifacts:
- ArtifactPath: 'example/path'
ArtifactName: 'exampleName'
- ArtifactPath: 'example/path'
ArtifactName: 'exampleName'
WebProject: true

I am afraid setting the default value for one of the properties cannot be done in the parameters setup.
But you can use the coalesce expression to check if the WebProject parameter is set or not:
Evaluates the parameters in order, and returns the value that does not equal null or empty-string.
Min parameters:
Max parameters: N
Example: coalesce(variables.couldBeNull, variables.couldAlsoBeNull, 'literal so it always works')
So you can use the expression coalesce(artifact.WebProject, false) in your template yaml as below:
- ${{ if eq(parameters.ArtifactPublish, true) }}:
- ${{ each artifact in parameters.Artifacts }}:
- ${{ if eq(coalesce(artifact.WebProject, false),false) }}:
With this expression - ${{ if eq(coalesce(artifact.WebProject, false),false) }}, No matter whether parameter WebProject is set to false or not used, this expression will always be true.

Related

Azure pipeline yml - Change value in task and execute based on if else condition

In the below task, based on the branch, how do I specify a different value for assembleQaDebug. If the branch is qa it has to be assembleQaDebug; if develop, it has to be assembleDevelopDebug . And if prod branch, it has to be assembleProdRelease. What is the best approach doing this, if there is one. Any help is much appreciated.
- task: Gradle#3
displayName: 'Build Task'
continueOnError: false
inputs:
tasks: assembleQaDebug -PversionCode=$(Build.BuildId) -PdisablePreDex --no-daemon
publishJUnitResults: false
Not sure if it's the best way to do this, but I love using variables for those kind of use cases, so check this one :
...
variables:
- name: assembleDebug
${{ if eq(variables['Build.SourceBranch'], 'refs/heads/prod') }}:
value: assembleProdRelease
${{ elseif eq(variables['Build.SourceBranch'], 'refs/heads/qa') }}:
value: assembleQaDebug
${{ elseif eq(variables['Build.SourceBranch'], 'refs/heads/develop') }}:
value: assembleDevelopDebug
...
- task: Gradle#3
displayName: 'Build Task'
continueOnError: false
inputs:
tasks: $(assembleDebug) -PversionCode=$(Build.BuildId) -PdisablePreDex --no-daemon
publishJUnitResults: false
The values has to be specified in 'variables' section:
variables:
- name: tf_version
value: latest
- name: variablegroupname
${{ if eq(variables['Build.SourceBranchName'], 'develop') }}:
value: assembleDevelopDebug
${{ if eq(variables['Build.SourceBranchName'], 'qa') }}:
value: assembleQaDebug
- name: env
${{ if eq(variables['Build.SourceBranchName'], 'main') }}:
value: prod
${{ if eq(variables['Build.SourceBranchName'], 'dev') }}:
value: develop
Like this you can add your variables and depending on your stage you can replace the values.

How do I reference a different Variable group per stage - stage per environment in environments - Azure Devops, template, variable groups

When I run the pipeline below I get an error within jobs.yml - it commplains about an "unexpected value" for the environment parameter... and within stages.yml "unexpected value: parameters" - what am I doing wrong here? The idea is to get the environments and the associated variableGroup - loop thru the environments array in Stages.yml and create a stage per environment... insert correct variableGroup per stage... use variablesGroup values to perform jobs in jobs.yml - each variable group contains the same vars.. with different values.
This is main.yml
#main.yml
trigger: none
extends:
template: /Build/Stages.yml
parameters:
environments:
- environment: Dev
variableGroup: Project-Dev-VarGrp
- environment: QA
variableGroup: Project-QA-VarGrp
dependsOn: Dev
- environment: UAT
variableGroup: Project-UAT-VarGrp
dependsOn: QA
- environment: UAT2
variableGroup: Project-UAT2-VarGrp
dependsOn: UAT
Then here is the next bit... Stages.yml
parameters:
- name: dataFolder
displayName: 'Data folder to process'
type: string
default: '/DataMigrations/Master_Data/'
- name: dataFiles
displayName: List of Data Files or Folder names
type: string
default: 'Dev_Data.zip'
- name: environments
type: object
default: []
stages:
- ${{ each environment in parameters.environments }}:
- stage: '${{ parameters.environment }}'
jobs:
- template: ./jobs.yml
variables:
- group: ${{ parameters.variableGroup }}
parameters:
environment: '${{ parameters.environment }}'
crmURL: $(crmURL)
oauthAppId: $(ClientID)
ClientPass: $(ClientPass)
dataFolder: '${{ parameters.dataFolder }}'
env: '${{ parameters.environment }}'
and here is jobs.yml
jobs:
- deployment: DeployData
displayName: 'Deploying Data to ${{ parameters.environment }}'
environment: ${{ parameters.environment }}
pool:
vmImage: 'windows-latest'
strategy:
runOnce:
deploy:
steps:
- checkout: self
clean: false
- powershell: |
Write-Host "##vso[task.setvariable variable=crmConnectionString]'AuthType=ClientSecret;Url=$(crmURL);ClientId=$(ClientID);ClientSecret=$(ClientPass)'"
displayName: 'PreDeploy configuration'
- task: PowerShell#2
displayName: 'Powershell: Run update-data.ps1'
inputs:
targetType: 'filePath'
filePath: $(System.DefaultWorkingDirectory)/DataMigrations/Scripts/update-data.ps1
arguments: -folderName '${{ parameters.dataFolder }}' -environment '${{ parameters.env }}'
workingDirectory: $(System.DefaultWorkingDirectory)/DataMigrations
- task: PowerShell#2
displayName: 'Powershell: Run zip-import-data.ps1'
inputs:
targetType: 'filePath'
filePath: $(System.DefaultWorkingDirectory)/DataMigrations/Scripts/zip-import-data.ps1
arguments: -folderName '${{ parameters.dataFolder }}' -dataMigrationFilenames '${{ parameters.dataFiles }}' -connectionString $(crmConnectionString)
workingDirectory: $(System.DefaultWorkingDirectory)/DataMigrations
Few things here:
When you're using ${{ each environment in parameters.environments }} then the nested environment and variableGroup are available using this syntax ${{ environment.environment }} and ${{ environment.variableGroup}}
In your Stages.yml file, you're trying to invoke the ./jobs.yml template but the associated parameters are defined after the - group: ${{ parameters.variableGroup }}. A valid syntax should looks like this:
stages:
- ${{ each environment in parameters.environments }}:
- stage: '${{ environment.environment }}'
variables:
- group: ${{ environment.variableGroup }}
jobs:
- template: ./jobs.yml
parameters:
environment: '${{ environment.environment }}'
crmURL: $(crmURL)
oauthAppId: $(ClientID)
ClientPass: $(ClientPass)
dataFolder: '${{ parameters.dataFolder }}'
env: '${{ environment.environment }}'
You also have few space typos. I know it's annoying but you need to have the exact yaml syntax otherwise the files can't be parsed.

azurepipelines template task condition not evaluating as expected

When I remove the task condition from the deploy_data_factory_steps.yaml template the parameter displays with the expected value:
parameter is DEV2
But when I run with the condition it fails to evaluate as True:
Evaluating: and(succeeded(), eq('$(env)', 'DEV2'))
Expanded: and(True, eq('$(env)', 'DEV2'))
Result: False
azurepipelines.yaml:
- stage: DataFactoryDEV2
condition: and(succeeded(), eq('${{ parameters.runEnv }}', 'dev2'))
dependsOn: DeployDEV2
variables:
- group: xxx_DEV2
pool:
name: 'yyyDev'
jobs:
- deployment: deploy
environment: xxx-dev2
strategy:
runOnce:
deploy:
steps:
- template: templates/deploy_data_factory_steps.yml
parameters:
environ: $(env)
artifact: 'publish-factory'
azureSubscription: 'yyy - xxx'
deploy_data_factory_steps.yaml
steps:
...
- task: AzureCLI#2
condition: and(succeeded(), eq('${{ parameters.environ }}', 'DEV2'))
displayName: 'Blah blah blah'
inputs:
azureSubscription: ${{ parameters.azureSubscription }}
scriptLocation: inlineScript
scriptType: bash
inlineScript: |
echo parameter is ${{ parameters.environ }}

Pipeline Artifact not Downloading with Stages

I have recently upgraded my YAML pipeline to include Stages, since this has happened the build artifacts are not downloading in a later stage task. For the life of me I cant figure out why.
Please see my YAML Code Bellow with an explanation on how my pipeline works.
First I have the main pipeline that calls the template yaml files.
mainpipeline.yml
pool:
vmImage: 'ubuntu-latest'
resources:
repositories:
- repository: Terraform
name: VALUE/Terraform
path:
- include: /Terraform
type: git
ref: main #branch name
- repository: Website
name: VALUE/Website
path:
- include: /Website
type: git
ref: newartifactpipeline #branch name
- repository: AuthenticationServer
name: VALUE/AuthenticationServer
path:
- include: /AuthenticationServer
type: git
ref: VALUE #branch name
trigger:
branches:
include:
- master
variables:
buildConfiguration: 'Release'
stages:
- stage: build_website_files
displayName: Building Main Website Files
jobs:
- job: build_main_website
steps:
- checkout: Website
- template: buildartifact.yml#website
parameters:
#vmImage: 'windows-latest'
buildConFiguration: $(buildConfiguration)
project: Website.csproj
artifactName: Website
- stage: build_authenticationserver_files
displayName: Building AuthenticationServer Website Files
jobs:
- job: build_authenticationserver_website
steps:
- checkout: AuthenticationServer
- template: buildartifact.yml#AuthenticationServer
parameters:
# vmImage: 'windows-latest'
buildConFiguration: $(buildConfiguration)
project: AuthenticationServer.csproj
artifactName: AuthenticationServer
- stage: run_terraform_pre_build
displayName: Building Terraform Applications and Deploying Web Apps
jobs:
- job: building_terraform_applications
steps:
- checkout: Terraform
- template: /VALUE/runterraform.yml#Terraform
parameters:
terraformWorkingDirectory: '$(System.DefaultWorkingDirectory)/VALUE'
serviceConnection: 'VALUE'
azureSubscription: 'VALUE'
appconnectionname: 'VALUE'
backendresourcegroupname: 'VALUE'
backendstorageaccountname: 'VALUE'
backendcontainername: 'VALUE'
RG: 'RG_Example'
azureLocation: 'UK South'
terraformVersion: '1.0.4'
artifactName: 'Website'
- stage: run_terraform_post_build
displayName: Apply Post Build Settings
jobs:
- job: apply_post_build_settings
steps:
- checkout: Terraform
- template: /Terraform/PostBuild/runterraformpostbuild.yml#Terraform
parameters:
terraformWorkingDirectory: '$(System.DefaultWorkingDirectory)/VALUE/PostBuild'
serviceConnection: 'VALUE'
azureSubscription: 'VALUE'
appconnectionname: 'VALUE'
backendresourcegroupname: ''VALUE''
backendstorageaccountname: 'VALUE'
backendcontainername: 'VALUE'
RG: 'RG_Example'
azureLocation: 'UK South'
terraformVersion: '1.0.4'
artifactName: 'Website'
The first stage builds and calls this build artifact template yaml file, that does successfully publish the artifact file, yaml code below:
Please see screenshot bellow as proof:
buildartifact.yml
parameters:
- name: buildConfiguration
type: string
default: 'Release'
- name: project
type: string
default: 'Website.csproj'
- name: artifactName
type: string
default: 'Website'
- name: vmImage
type: string
default: 'windows-latest'
jobs:
- job: build_website
pool:
vmImage: ${{ parameters.vmImage }}
steps:
- checkout: Website
- task: CmdLine#2
inputs:
script: |
echo '$(System.DefaultWorkingDirectory)'
dir
- task: DotNetCoreCLI#2
displayName: dotnet restore
inputs:
command: restore
projects: '**/${{ parameters.project }}'
# Node.js tool installer
# Finds or downloads and caches the specified version spec of Node.js and adds it to the PATH
- task: NodeTool#0
displayName: 'Install Node .js'
inputs:
versionSpec: '14.17.3'
force32bit: false # Optional
checkLatest: false # Optional
- script: |
npm install -g #angular/cli#12.1.3
npm install
displayName: 'npm install'
- task: Npm#1
displayName: 'npm run build'
inputs:
command: 'custom'
workingDir: ClientApp
customCommand: 'build'
- task: DotNetCoreCLI#2
displayName: 'Build'
inputs:
command: 'build'
projects: '**/${{ parameters.project }}'
arguments: '--configuration ${{ parameters.buildConfiguration }}'
- task: DotNetCoreCLI#2
displayName: dotnet restore unit tests
inputs:
command: restore
projects: 'UnitTests/UnitTests.csproj'
- task: DotNetCoreCLI#2
displayName: dotnet Test
inputs:
command: test
projects: 'UnitTests/UnitTests.csproj'
arguments: '--configuration Release'
- task: DotNetCoreCLI#2
displayName: 'Publish Application'
inputs:
command: 'publish'
publishWebProjects: false
projects: '**/${{ parameters.project }}'
arguments: '--configuration ${{ parameters.buildConfiguration }} --output $(Pipeline.Workspace)/website/'
- task: PublishPipelineArtifact#1
displayName: 'Publish Artifacts'
inputs:
targetPath: '$(Pipeline.Workspace)/website/'
artifact: ${{ parameters.artifactName }}
publishLocation: 'pipeline'
With this yaml for the PublishPipelineArtifact task I have tried the following env variables for pipleines: pipeline.workspace and System.DefaultWorkingDirectory
Yet both have not worked in the later stage where the final yaml file tries to download the pipeline artifact see yaml code bellow:
runterraformanddownloadartifact.yml
parameters:
- name: terraformWorkingDirectory
type: string
default: $(System.DefaultWorkingDirectory)/Terraform
- name: serviceConnection
type: string
default: value
- name: azureSubscription
type: string
default: value
- name: appconnectionname
type: string
default: value
- name: backendresourcegroupname
type: string
default: DevOpsTerraform
- name: backendstorageaccountname
type: string
default: value
- name: backendcontainername
type: string
default: value
- name: RG
type: string
default: RG_Example
- name: azureLocation
type: string
default: UK South
- name: terraformVersion
type: string
default: 1.0.4
- name: artifactName
type: string
default: Website
jobs:
- job: Run_Terraform
displayName: Installing and Running Terraform
steps:
- checkout: Terraform
- task: TerraformInstaller#0
displayName: install
inputs:
terraformVersion: '${{ parameters.terraformVersion }}'
- task: CmdLine#2
inputs:
script: |
echo '$(System.DefaultWorkingDirectory)'
dir
- task: TerraformTaskV2#2
displayName: init
inputs:
provider: azurerm
command: init
backendServiceArm: '${{ parameters.serviceConnection }}'
backendAzureRmResourceGroupName: '${{ parameters.backendresourcegroupname }}'
backendAzureRmStorageAccountName: '${{ parameters.backendstorageaccountname }}'
backendAzureRmContainerName: '${{ parameters.backendcontainername }}'
backendAzureRmKey: terraform.tfstate
workingDirectory: '${{ parameters.terraformWorkingDirectory }}'
- task: TerraformTaskV1#0
displayName: plan
inputs:
provider: azurerm
command: plan
commandOptions: '-input=false'
environmentServiceNameAzureRM: '${{ parameters.serviceConnection }}'
workingDirectory: '${{ parameters.terraformWorkingDirectory }}'
- task: TerraformTaskV1#0
displayName: apply
inputs:
provider: azurerm
command: apply
commandOptions: '-input=false -auto-approve'
environmentServiceNameAzureRM: '${{ parameters.serviceConnection }}'
workingDirectory: '${{ parameters.terraformWorkingDirectory }}'
- job: Put_artifacts_into_place
displayName: Putting_artifacts_into_place
dependsOn: Run_Terraform
steps:
- checkout: Website
- checkout: AuthenticationServer
- task: DownloadPipelineArtifact#2
displayName: Download Build Artifacts
inputs:
artifact: '${{ parameters.artifactName }}'
patterns: /website/**/*.zip
path: $(Pipeline.Workspace)/website/
- task: AzureWebApp#1
displayName: 'Azure Web App Deploy: VALUE'
inputs:
package: $(Pipeline.Workspace)/website/**/*.zip
azureSubscription: '${{ parameters.azureSubscription }}'
ConnectedServiceName: '${{ parameters.appconnectionname}}'
appName: VALUE
ResourceGroupName: '${{ parameters.RG}}'
- task: DownloadPipelineArtifact#2
displayName: Download Build Artifacts
inputs:
artifact: '${{ parameters.artifactName}}'
patterns: /authsrv/**/*.zip
path: $(Pipeline.Workspace)/authsrv/
- task: AzureWebApp#1
displayName: 'Azure Web App Deploy: VALUE'
inputs:
package: $(Pipeline.Workspace)/authsrv/**/*.zip
azureSubscription: '${{ parameters.azureSubscription }}'
ConnectedServiceName: '${{ parameters.appconnectionname}}'
appName: VALUE
ResourceGroupName: '${{ parameters.RG}}'
Essentially the first pipeline calls these two pipelines which are wrapped in Stages. Before they were not wrapped in stages and this pipeline worked well. Since moving it to stages I have had the issue where the task: DownloadPipelineArtifact#2 completes but downloads nothing. Please see screen shot bellow:
The error I am getting at the end of the pipeline is:
##[error]Error: No package found with specified pattern: /home/vsts/work/1/website/**/*.zip<br/>Check if the package mentioned in the task is published as an artifact in the build or a previous stage and downloaded in the current job.
I have tried the following solutions without success:
File pattern for Publish Pipeline Artifact in Azure DevOps
how to use PublishPipelineArtifact#1 with build script
And consulted MS Doc: https://learn.microsoft.com/en-us/azure/devops/pipelines/artifacts/pipeline-artifacts?view=azure-devops&tabs=yaml

Why my multi stage full yaml azure devops pipeline breaks when moved to template?

I have a multi stage full yaml pipeline in azure devops services. The different stages consist of a build stage, a Dev stage (Deploys azure resources to an azure resource group) and a UAT stage (Deploys azure resources to a different resource group). Everything works great. However I needed to use templates because I want to reuse the steps. After moving the steps to a template, the dev stage keeps working but the UAT stage gives authorization error. It is worth mentioning, dev and uat stages use different service connections targeting different subscriptions. But why will it work when I dont use templates and simply moving the steps to a template file all of a sudden it doesn't work. Is this a known bug?
This is the error I get when using the template approach:
| The client '9e5cc21a-bb38-46b9-a16a-289fbbf9c8b9' with object
| id '9e5cc21a-bb38-46b9-a16a-289fbbf9c8b9' does not have
| authorization to perform action
| 'Microsoft.Resources/subscriptions/resourceGroups/resources/read' over scope '/subscriptions/7041f5ba-1040-4989-8e48-497b3b826d01/resourceGroups/Resource-Group-Test-A' or the scope is invalid. If access was recently granted, please refresh your credentials. StatusCode: 403 ReasonPhrase: Forbidden OperationID : 464f577e-6617-4bed-9a14-1f7487b5f209
This is the pipeline without using templates (works perfect!).
# Starter pipeline
# Start with a minimal pipeline that you can customize to build and deploy your code.
# Add steps that build, run tests, deploy, and more:
# https://aka.ms/yaml
trigger:
- master
variables:
- group: 'Dev ARMT SFTP Connection'
resources:
repositories:
- repository: templates
type: git
name: CommonTasks
stages:
- stage: Build
jobs:
- job: Build
pool:
vmImage: 'ubuntu-latest'
steps:
- task: printAllVariables#1
- task: CopyFiles#2
inputs:
SourceFolder: '$(System.DefaultWorkingDirectory)'
Contents: 'ARM-Templates/**'
TargetFolder: '$(Build.ArtifactStagingDirectory)'
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)'
ArtifactName: 'drop'
publishLocation: 'Container'
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
# Write your PowerShell commands here.
Write-Host "Build Completed..."
$workingdir = "$(Build.ArtifactStagingDirectory)"
Write-Host $workingdir
$fcontent = Get-ChildItem -Path $workingdir
Write-Host $fcontent
- stage: Dev
jobs:
- job: Dev
pool:
vmImage: 'ubuntu-latest'
steps:
- task: DownloadBuildArtifacts#0
inputs:
buildType: 'specific'
project: '530cfd4b-51b8-4237-b2fa-f296a4cba29d'
pipeline: '30'
buildVersionToDownload: 'latest'
downloadType: 'single'
artifactName: 'drop'
downloadPath: '$(System.ArtifactsDirectory)/CommonTasks'
- task: DownloadBuildArtifacts#0
inputs:
buildType: 'current'
downloadType: 'single'
artifactName: 'drop'
downloadPath: '$(System.ArtifactsDirectory)/SftpConnection'
- task: FileTransform#1
displayName: 'ARMT SFTP Connection - Parameters File Transform '
inputs:
folderPath: '$(System.ArtifactsDirectory)/SftpConnection/drop'
fileType: json
targetFiles: 'ARM-Templates/parameters.json'
- task: AzurePowerShell#4
displayName: 'ARMT SFTP Connection - Check Resource Existance'
inputs:
azureSubscription: 'Starwood-DT-DEV-ServiceConnection'
ScriptPath: '$(System.ArtifactsDirectory)/CommonTasks/drop/AzurePowerShell/ResourceExistance.ps1'
ScriptArguments: '-resourceGroupName Starwood-DT-DEV -resourceName $(parameters.sftp_name.value)'
azurePowerShellVersion: LatestVersion
- task: AzureResourceGroupDeployment#2
displayName: 'ARMT SFTP Connection - Deploy'
inputs:
azureSubscription: 'Starwood-DT-DEV-ServiceConnection'
resourceGroupName: 'Starwood-DT-DEV'
location: 'East US'
csmFile: '$(System.ArtifactsDirectory)/SftpConnection/drop/ARM-Templates/template.json'
csmParametersFile: '$(System.ArtifactsDirectory)/SftpConnection/drop/ARM-Templates/parameters.json'
condition: eq(variables['deployresource'],'true')
- stage: GroupATest
jobs:
- job: GroupATest
pool:
vmImage: 'ubuntu-latest'
steps:
- task: DownloadBuildArtifacts#0
inputs:
buildType: 'specific'
project: '530cfd4b-51b8-4237-b2fa-f296a4cba29d'
pipeline: '30'
buildVersionToDownload: 'latest'
downloadType: 'single'
artifactName: 'drop'
downloadPath: '$(System.ArtifactsDirectory)/CommonTasks'
- task: DownloadBuildArtifacts#0
inputs:
buildType: 'current'
downloadType: 'single'
artifactName: 'drop'
downloadPath: '$(System.ArtifactsDirectory)/SftpConnection'
- task: FileTransform#1
displayName: 'ARMT SFTP Connection - Parameters File Transform '
inputs:
folderPath: '$(System.ArtifactsDirectory)/SftpConnection/drop'
fileType: json
targetFiles: 'ARM-Templates/parameters.json'
- task: AzurePowerShell#4
displayName: 'ARMT SFTP Connection - Check Resource Existance'
inputs:
azureSubscription: 'Resource-Group-Test-A'
ScriptPath: '$(System.ArtifactsDirectory)/CommonTasks/drop/AzurePowerShell/ResourceExistance.ps1'
ScriptArguments: '-resourceGroupName Resouce-Group-Test-A -resourceName $(parameters.sftp_name.value)'
azurePowerShellVersion: LatestVersion
- task: AzureResourceGroupDeployment#2
displayName: 'ARMT SFTP Connection - Deploy'
inputs:
azureSubscription: 'Resource-Group-Test-A'
resourceGroupName: 'Resouce-Group-Test-A'
location: 'East US'
csmFile: '$(System.ArtifactsDirectory)/SftpConnection/drop/ARM-Templates/template.json'
csmParametersFile: '$(System.ArtifactsDirectory)/SftpConnection/drop/ARM-Templates/parameters.json'
condition: eq(variables['deployresource'],'true')
When moving to steps template (then i get the error):
# Starter pipeline
# Start with a minimal pipeline that you can customize to build and deploy your code.
# Add steps that build, run tests, deploy, and more:
# https://aka.ms/yaml
trigger:
- master
resources:
repositories:
- repository: templates
type: git
name: CommonTasks
stages:
- stage: Build
jobs:
- job: Build
pool:
vmImage: 'ubuntu-latest'
steps:
- task: printAllVariables#1
- task: CopyFiles#2
inputs:
SourceFolder: '$(System.DefaultWorkingDirectory)'
Contents: 'ARM-Templates/**'
TargetFolder: '$(Build.ArtifactStagingDirectory)'
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)'
ArtifactName: 'drop'
publishLocation: 'Container'
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
# Write your PowerShell commands here.
Write-Host "Build Completed..."
$workingdir = "$(Build.ArtifactStagingDirectory)"
Write-Host $workingdir
$fcontent = Get-ChildItem -Path $workingdir
Write-Host $fcontent
- stage: Dev
variables:
- group: 'Dev ARMT SFTP Connection'
jobs:
- job: Dev
pool:
vmImage: 'ubuntu-latest'
steps:
- task: DownloadBuildArtifacts#0
inputs:
buildType: 'specific'
project: '530cfd4b-51b8-4237-b2fa-f296a4cba29d'
pipeline: '30'
buildVersionToDownload: 'latest'
downloadType: 'single'
artifactName: 'drop'
downloadPath: '$(System.ArtifactsDirectory)/CommonTasks'
- task: DownloadBuildArtifacts#0
inputs:
buildType: 'current'
downloadType: 'single'
artifactName: 'drop'
downloadPath: '$(System.ArtifactsDirectory)/SftpConnection'
- template: YamlTemplate/azure-resource-deploy.yml#templates
parameters:
dropLocation: '$(System.ArtifactsDirectory)/SftpConnection/drop'
transformTargetPath: 'ARM-Templates/parameters.json'
resourceName: $(parameters.sftp_name.value)
resourceGroupName: 'Starwood-DT-DEV'
azureServiceConnectionName: 'Starwood-DT-DEV-ServiceConnection'
resourceLocation: 'East US'
armtTemplateFilePath: '$(System.ArtifactsDirectory)/SftpConnection/drop/ARM-Templates/template.json'
armtParemeterFilePath: '$(System.ArtifactsDirectory)/SftpConnection/drop/ARM-Templates/parameters.json'
- stage: GroupATest
variables:
- group: 'GroupA ARMT SFTP Connection'
jobs:
- job: GroupATest
pool:
vmImage: 'ubuntu-latest'
steps:
- task: DownloadBuildArtifacts#0
inputs:
buildType: 'specific'
project: '530cfd4b-51b8-4237-b2fa-f296a4cba29d'
pipeline: '30'
buildVersionToDownload: 'latest'
downloadType: 'single'
artifactName: 'drop'
downloadPath: '$(System.ArtifactsDirectory)/CommonTasks'
- task: DownloadBuildArtifacts#0
inputs:
buildType: 'current'
downloadType: 'single'
artifactName: 'drop'
downloadPath: '$(System.ArtifactsDirectory)/SftpConnection'
- template: YamlTemplate/azure-resource-deploy.yml#templates
parameters:
dropLocation: '$(System.ArtifactsDirectory)/SftpConnection/drop'
transformTargetPath: 'ARM-Templates/parameters.json'
resourceName: $(parameters.sftp_name.value)
resourceGroupName: 'Resource-Group-Test-A'
azureServiceConnectionName: 'Resource-Group-Test-A'
resourceLocation: 'East US'
armtTemplateFilePath: '$(System.ArtifactsDirectory)/SftpConnection/drop/ARM-Templates/template.json'
armtParemeterFilePath: '$(System.ArtifactsDirectory)/SftpConnection/drop/ARM-Templates/parameters.json'
This the template structure
Parent template - (azure-resource-deploy.yml):
parameters:
- name: dropLocation
type: string
displayName: 'Artifact path to drop location'
default: ''
- name: transformTargetPath
type: string
default: ''
- name: resourceName
type: string
default: ''
- name: resourceGroupName
type: string
default: ''
- name: azureServiceConnectionName
type: string
default: ''
- name: resourceLocation
type: string
default: ''
- name: armtTemplateFilePath
type: string
default: ''
- name: armtParemeterFilePath
type: string
default: ''
steps:
- template: armt-parameter-file-transform.yml
parameters:
transformStepDisplayName: ''
folderPath: ${{parameters.dropLocation}}
targetFile: ${{parameters.transformTargetPath}}
- template: azure-resource-check.yml
parameters:
resourceName: ${{parameters.resourceName}}
resourceGroupName: ${{parameters.resourceGroupName}}
azureServiceConnectionName: ${{parameters.azureServiceConnectionName}}
- template: armt-deploy.yml
parameters:
resourceName: ${{parameters.resourceName}}
resourceGroupName: ${{parameters.resourceGroupName}}
resourceLocation: ${{parameters.resourceLocation}}
azureServiceConnectionName: ${{parameters.azureServiceConnectionName}}
templateFilePath: ${{parameters.armtTemplateFilePath}}
parametersFilePath: ${{parameters.armtParemeterFilePath}}
Templates referenced from parent:
(armt-parameter-file-transform.yml)
parameters:
- name: transformStepDisplayName
type: string
displayName: 'Display name for this step'
default: ''
- name: folderPath
type: string
displayName: 'Path to drop location'
default: ''
- name: targetFile
type: string
displayName: 'Path to paremeter file relative to drop location.'
default: ''
steps:
- task: FileTransform#1
displayName: ${{parameters.transformStepDisplayName}}
inputs:
folderPath: ${{parameters.folderPath}}
fileType: json
targetFiles: ${{parameters.targetFile}}
(azure-resource-check.yml)
parameters:
- name: resourceName
type: string
- name: resourceGroupName
type: string
- name: azureServiceConnectionName
type: string
steps:
- script: echo Echo -resourceGroupName ${{ parameters.resourceGroupName }} -resourceName ${{ parameters.resourceName }}
- task: AzurePowerShell#4
displayName: '${{ parameters.resourceName }} - Checking Resource Existance'
inputs:
azureSubscription: '${{ parameters.azureServiceConnectionName }}'
ScriptPath: '$(System.ArtifactsDirectory)/CommonTasks/drop/AzurePowerShell/ResourceExistance.ps1'
ScriptArguments: '-resourceGroupName ${{ parameters.resourceGroupName }} -resourceName ${{ parameters.resourceName }}'
azurePowerShellVersion: LatestVersion
(armt-deploy.yml)
parameters:
- name: resourceName
type: string
- name: resourceGroupName
type: string
- name: resourceLocation
type: string
- name: azureServiceConnectionName
type: string
- name: templateFilePath
type: string
- name: parametersFilePath
type: string
steps:
- task: AzureResourceGroupDeployment#2
displayName: 'ARMT Deploy - ${{parameters.resourceName}}'
inputs:
azureSubscription: ${{parameters.azureServiceConnectionName}}
resourceGroupName: ${{parameters.resourceGroupName}}
location: ${{parameters.resourceLocation}}
csmFile: ${{parameters.templateFilePath}}
csmParametersFile: ${{parameters.parametersFilePath}}
condition: eq(variables['deployresource'],'true')
This was just caused by a typo. The actual name of the resource group is Resouce-Group-Test-A, I missed the r when naming this resource group. The error does not happen in the non-templated version because when you pick the resource group you do it from a drop down list, with the correct prepopulated names, so no chance to make a mistake. However in a full yaml pipeline you have to type it in, and thats where the typo occured. If there is anything to be learned here. Pay close attention when typing resource names, errors might not be descriptive enough. Sorry If wasted anybodys precious time.

Resources