Trying to deploy ionic5 angular8 pwa to aws ec2 from azure devops YAML - azure

I am new to azure devops.I am trying to deploy my ionic5 angular application on amazon ec2 via azure pipelines.Whenever i am trying to run the pipeline it is giving "Encountered error(s) while parsing pipeline YAML:
/build-ci.yml (Line: 2, Col: 3): A mapping was not expected" this error.
This is yaml file code.I am quite stucked here. Please help.
trigger:
- task: CodeDeployDeployApplication#1
inputs:
awsCredentials: 'AWS Service Con'
regionName: 'us-east-1'
applicationName: 'Dev-erp-Frontend'
deploymentGroupName: 'Dev-erp-Frontend'
deploymentRevisionSource: 'workspace'
revisionBundle: 'Dev-erp-Frontend-Rev'
bucketName: 'Dev-erp-Frontend'
fileExistsBehavior: 'OVERWRITE'
batch: "true"
branches:
include:
- master
- dev
paths:
include:
- ./*
pr:
branches:
include:
- master
- dev
paths:
include:
- ./*
jobs:
- job: Build_Job
displayName: Build
pool:
vmImage: 'ubuntu-18.04'
demands:
- npm
steps:
- checkout: self
clean: false
# - powershell: 'npm cache clean --force'
# displayName: 'PowerShell Script'
# env:
# APPDATA: npm-cache
- task: Npm#1
displayName: 'Npm Install'
inputs:
workingDir: "./"
command: "ci"
- task: Npm#1
displayName: 'Lint Client App'
inputs:
workingDir: "./"
command: "custom"
customCommand: "run lint"
continueOnError: true
- task: Npm#1
displayName: 'Copy Assets'
inputs:
workingDir: "./"
command: "custom"
customCommand: "run copy-files"
continueOnError: false
- task: Npm#1
displayName: 'Build Client App'
inputs:
workingDir: "./"
command: "custom"
customCommand: "run build:prod"
# Archive files
- task: ArchiveFiles#2
inputs:
rootFolderOrFile: '$(Build.BinariesDirectory)'
includeRootFolder: true
archiveType: 'zip' # Options: zip, 7z, tar, wim
archiveFile: '$(Build.ArtifactStagingDirectory)/www.zip'
replaceExistingArchive: true
- task: CopyFiles#2
inputs:
Contents: 'www/**'
TargetFolder: '$(build.artifactstagingdirectory)'
- task: PublishBuildArtifacts#1
displayName: 'Publish Artifact'
inputs:
PathtoPublish: '$(build.artifactstagingdirectory)'
ArtifactName: 'titas-ecom-erp'

We cannot set the task CodeDeployDeployApplication#1 at the trigger level, we should add the task at the steps level. You can update your YAML definition as follows.
trigger:
branches:
include:
- master
- dev
paths:
include:
- ./*
pr:
branches:
include:
- master
- dev
paths:
include:
- ./*
jobs:
- job: Build_Job
displayName: Build
pool:
vmImage: 'ubuntu-18.04'
demands:
- npm
steps:
- checkout: self
clean: false
# - powershell: 'npm cache clean --force'
# displayName: 'PowerShell Script'
# env:
# APPDATA: npm-cache
- task: CodeDeployDeployApplication#1
inputs:
awsCredentials: 'AWS Service Con'
regionName: 'us-east-1'
applicationName: 'Dev-erp-Frontend'
deploymentGroupName: 'Dev-erp-Frontend'
deploymentRevisionSource: 'workspace'
revisionBundle: 'Dev-erp-Frontend-Rev'
bucketName: 'Dev-erp-Frontend'
fileExistsBehavior: 'OVERWRITE'
batch: "true"
- task: Npm#1
displayName: 'Npm Install'
inputs:
workingDir: "./"
command: "ci"
- task: Npm#1
displayName: 'Lint Client App'
inputs:
workingDir: "./"
command: "custom"
customCommand: "run lint"
continueOnError: true
- task: Npm#1
displayName: 'Copy Assets'
inputs:
workingDir: "./"
command: "custom"
customCommand: "run copy-files"
continueOnError: false
- task: Npm#1
displayName: 'Build Client App'
inputs:
workingDir: "./"
command: "custom"
customCommand: "run build:prod"
# Archive files
- task: ArchiveFiles#2
inputs:
rootFolderOrFile: '$(Build.BinariesDirectory)'
includeRootFolder: true
archiveType: 'zip' # Options: zip, 7z, tar, wim
archiveFile: '$(Build.ArtifactStagingDirectory)/www.zip'
replaceExistingArchive: true
- task: CopyFiles#2
inputs:
Contents: 'www/**'
TargetFolder: '$(build.artifactstagingdirectory)'
- task: PublishBuildArtifacts#1
displayName: 'Publish Artifact'
inputs:
PathtoPublish: '$(build.artifactstagingdirectory)'
ArtifactName: 'titas-ecom-erp'

Related

Azure multistage yaml pipeline how to substitute appsetting indexed value

I have an issue relating to appsettings.json substitution in yaml variable.
app setting config
"Serilog": {
"LevelSwitches": { "$controlSwitch": "Information" },
"MinimumLevel": {
"ControlledBy": "$controlSwitch",
"Override": {
"Microsoft": "Warning",
"System": "Warning"
}
},
"WriteTo": [
{
"Name": "Seq",
"Args": {
"serverUrl": "!FromSecret!",
"apiKey": "!FromSecret!",,
"controlLevelSwitch": "$controlSwitch"
}
}
],
"Enrich": [ "FromLogContext", "WithMachineName", "WithProcessId", "WithThreadId" ],
"Properties": {
"Application": ""
}
in yaml I need to replace !FromSecret! with actual value
variables:
- name: Serilog.WriteTo.0.Args.apiKey
value: '$(Secret-AppSettings-Serilog-ApiKey)'
- name: Serilog.WriteTo.0.Args.serverUrl
value: '$(Secret-API-AppSettings-Serilog-Url)'
the above yaml substitution is now working as unable to find the index need a help
need expert help on this issue
You can use below yaml script to modify ARGS object in an array element and index like below:-
variables:
- name: Serilog.WriteTo[0].Args.apiKey
value: '$(Secret-AppSettings-Serilog-ApiKey)'
- name: Serilog.WriteTo[0].Args.serverUrl
value: '$(Secret-API-AppSettings-Serilog-Url)'
Complete YAML script with app settings and Azure Web app :-
trigger:
- main
variables:
solution: '**/*.sln'
buildPlatform: 'Any CPU'
buildConfiguration: 'Release'
appSettingsFile: 'path/to/appsettings.json'
serverUrlVariable: '$(Secret-API-AppSettings-Serilog-Url)'
apiKeyVariable: '$(Secret-AppSettings-Serilog-ApiKey)'
steps:
- task: DotNetCoreCLI#2
inputs:
command: 'restore'
projects: '$(solution)'
- task: DotNetCoreCLI#2
inputs:
command: 'build'
projects: '$(solution)'
arguments: '--configuration $(buildConfiguration)'
- task: DotNetCoreCLI#2
inputs:
command: 'publish'
projects: '$(solution)'
publishWebProjects: true
arguments: '--configuration $(buildConfiguration) --output $(build.artifactstagingdirectory)'
zipAfterPublish: true
- task: AzureAppConfiguration#1
inputs:
azureSubscription: 'My Azure Subscription'
action: 'SetKeyValue'
Key: 'Serilog:WriteTo:0:Args:serverUrl'
Value: '$(serverUrlVariable)'
AppConfigurations: 'MyAppConfig'
AppConfigurationEnvironment: 'MyAppConfigEnvironment'
- task: AzureAppConfiguration#1
inputs:
azureSubscription: 'My Azure Subscription'
action: 'SetKeyValue'
Key: 'Serilog:WriteTo:0:Args:apiKey'
Value: '$(apiKeyVariable)'
AppConfigurations: 'MyAppConfig'
AppConfigurationEnvironment: 'MyAppConfigEnvironment'
- task: CopyFiles#2
inputs:
SourceFolder: '$(System.DefaultWorkingDirectory)'
Contents: '**/appsettings.json'
TargetFolder: '$(Build.ArtifactStagingDirectory)'
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)'
ArtifactName: 'drop'

Pipeline Artifact not Downloading with Stages

I have recently upgraded my YAML pipeline to include Stages, since this has happened the build artifacts are not downloading in a later stage task. For the life of me I cant figure out why.
Please see my YAML Code Bellow with an explanation on how my pipeline works.
First I have the main pipeline that calls the template yaml files.
mainpipeline.yml
pool:
vmImage: 'ubuntu-latest'
resources:
repositories:
- repository: Terraform
name: VALUE/Terraform
path:
- include: /Terraform
type: git
ref: main #branch name
- repository: Website
name: VALUE/Website
path:
- include: /Website
type: git
ref: newartifactpipeline #branch name
- repository: AuthenticationServer
name: VALUE/AuthenticationServer
path:
- include: /AuthenticationServer
type: git
ref: VALUE #branch name
trigger:
branches:
include:
- master
variables:
buildConfiguration: 'Release'
stages:
- stage: build_website_files
displayName: Building Main Website Files
jobs:
- job: build_main_website
steps:
- checkout: Website
- template: buildartifact.yml#website
parameters:
#vmImage: 'windows-latest'
buildConFiguration: $(buildConfiguration)
project: Website.csproj
artifactName: Website
- stage: build_authenticationserver_files
displayName: Building AuthenticationServer Website Files
jobs:
- job: build_authenticationserver_website
steps:
- checkout: AuthenticationServer
- template: buildartifact.yml#AuthenticationServer
parameters:
# vmImage: 'windows-latest'
buildConFiguration: $(buildConfiguration)
project: AuthenticationServer.csproj
artifactName: AuthenticationServer
- stage: run_terraform_pre_build
displayName: Building Terraform Applications and Deploying Web Apps
jobs:
- job: building_terraform_applications
steps:
- checkout: Terraform
- template: /VALUE/runterraform.yml#Terraform
parameters:
terraformWorkingDirectory: '$(System.DefaultWorkingDirectory)/VALUE'
serviceConnection: 'VALUE'
azureSubscription: 'VALUE'
appconnectionname: 'VALUE'
backendresourcegroupname: 'VALUE'
backendstorageaccountname: 'VALUE'
backendcontainername: 'VALUE'
RG: 'RG_Example'
azureLocation: 'UK South'
terraformVersion: '1.0.4'
artifactName: 'Website'
- stage: run_terraform_post_build
displayName: Apply Post Build Settings
jobs:
- job: apply_post_build_settings
steps:
- checkout: Terraform
- template: /Terraform/PostBuild/runterraformpostbuild.yml#Terraform
parameters:
terraformWorkingDirectory: '$(System.DefaultWorkingDirectory)/VALUE/PostBuild'
serviceConnection: 'VALUE'
azureSubscription: 'VALUE'
appconnectionname: 'VALUE'
backendresourcegroupname: ''VALUE''
backendstorageaccountname: 'VALUE'
backendcontainername: 'VALUE'
RG: 'RG_Example'
azureLocation: 'UK South'
terraformVersion: '1.0.4'
artifactName: 'Website'
The first stage builds and calls this build artifact template yaml file, that does successfully publish the artifact file, yaml code below:
Please see screenshot bellow as proof:
buildartifact.yml
parameters:
- name: buildConfiguration
type: string
default: 'Release'
- name: project
type: string
default: 'Website.csproj'
- name: artifactName
type: string
default: 'Website'
- name: vmImage
type: string
default: 'windows-latest'
jobs:
- job: build_website
pool:
vmImage: ${{ parameters.vmImage }}
steps:
- checkout: Website
- task: CmdLine#2
inputs:
script: |
echo '$(System.DefaultWorkingDirectory)'
dir
- task: DotNetCoreCLI#2
displayName: dotnet restore
inputs:
command: restore
projects: '**/${{ parameters.project }}'
# Node.js tool installer
# Finds or downloads and caches the specified version spec of Node.js and adds it to the PATH
- task: NodeTool#0
displayName: 'Install Node .js'
inputs:
versionSpec: '14.17.3'
force32bit: false # Optional
checkLatest: false # Optional
- script: |
npm install -g #angular/cli#12.1.3
npm install
displayName: 'npm install'
- task: Npm#1
displayName: 'npm run build'
inputs:
command: 'custom'
workingDir: ClientApp
customCommand: 'build'
- task: DotNetCoreCLI#2
displayName: 'Build'
inputs:
command: 'build'
projects: '**/${{ parameters.project }}'
arguments: '--configuration ${{ parameters.buildConfiguration }}'
- task: DotNetCoreCLI#2
displayName: dotnet restore unit tests
inputs:
command: restore
projects: 'UnitTests/UnitTests.csproj'
- task: DotNetCoreCLI#2
displayName: dotnet Test
inputs:
command: test
projects: 'UnitTests/UnitTests.csproj'
arguments: '--configuration Release'
- task: DotNetCoreCLI#2
displayName: 'Publish Application'
inputs:
command: 'publish'
publishWebProjects: false
projects: '**/${{ parameters.project }}'
arguments: '--configuration ${{ parameters.buildConfiguration }} --output $(Pipeline.Workspace)/website/'
- task: PublishPipelineArtifact#1
displayName: 'Publish Artifacts'
inputs:
targetPath: '$(Pipeline.Workspace)/website/'
artifact: ${{ parameters.artifactName }}
publishLocation: 'pipeline'
With this yaml for the PublishPipelineArtifact task I have tried the following env variables for pipleines: pipeline.workspace and System.DefaultWorkingDirectory
Yet both have not worked in the later stage where the final yaml file tries to download the pipeline artifact see yaml code bellow:
runterraformanddownloadartifact.yml
parameters:
- name: terraformWorkingDirectory
type: string
default: $(System.DefaultWorkingDirectory)/Terraform
- name: serviceConnection
type: string
default: value
- name: azureSubscription
type: string
default: value
- name: appconnectionname
type: string
default: value
- name: backendresourcegroupname
type: string
default: DevOpsTerraform
- name: backendstorageaccountname
type: string
default: value
- name: backendcontainername
type: string
default: value
- name: RG
type: string
default: RG_Example
- name: azureLocation
type: string
default: UK South
- name: terraformVersion
type: string
default: 1.0.4
- name: artifactName
type: string
default: Website
jobs:
- job: Run_Terraform
displayName: Installing and Running Terraform
steps:
- checkout: Terraform
- task: TerraformInstaller#0
displayName: install
inputs:
terraformVersion: '${{ parameters.terraformVersion }}'
- task: CmdLine#2
inputs:
script: |
echo '$(System.DefaultWorkingDirectory)'
dir
- task: TerraformTaskV2#2
displayName: init
inputs:
provider: azurerm
command: init
backendServiceArm: '${{ parameters.serviceConnection }}'
backendAzureRmResourceGroupName: '${{ parameters.backendresourcegroupname }}'
backendAzureRmStorageAccountName: '${{ parameters.backendstorageaccountname }}'
backendAzureRmContainerName: '${{ parameters.backendcontainername }}'
backendAzureRmKey: terraform.tfstate
workingDirectory: '${{ parameters.terraformWorkingDirectory }}'
- task: TerraformTaskV1#0
displayName: plan
inputs:
provider: azurerm
command: plan
commandOptions: '-input=false'
environmentServiceNameAzureRM: '${{ parameters.serviceConnection }}'
workingDirectory: '${{ parameters.terraformWorkingDirectory }}'
- task: TerraformTaskV1#0
displayName: apply
inputs:
provider: azurerm
command: apply
commandOptions: '-input=false -auto-approve'
environmentServiceNameAzureRM: '${{ parameters.serviceConnection }}'
workingDirectory: '${{ parameters.terraformWorkingDirectory }}'
- job: Put_artifacts_into_place
displayName: Putting_artifacts_into_place
dependsOn: Run_Terraform
steps:
- checkout: Website
- checkout: AuthenticationServer
- task: DownloadPipelineArtifact#2
displayName: Download Build Artifacts
inputs:
artifact: '${{ parameters.artifactName }}'
patterns: /website/**/*.zip
path: $(Pipeline.Workspace)/website/
- task: AzureWebApp#1
displayName: 'Azure Web App Deploy: VALUE'
inputs:
package: $(Pipeline.Workspace)/website/**/*.zip
azureSubscription: '${{ parameters.azureSubscription }}'
ConnectedServiceName: '${{ parameters.appconnectionname}}'
appName: VALUE
ResourceGroupName: '${{ parameters.RG}}'
- task: DownloadPipelineArtifact#2
displayName: Download Build Artifacts
inputs:
artifact: '${{ parameters.artifactName}}'
patterns: /authsrv/**/*.zip
path: $(Pipeline.Workspace)/authsrv/
- task: AzureWebApp#1
displayName: 'Azure Web App Deploy: VALUE'
inputs:
package: $(Pipeline.Workspace)/authsrv/**/*.zip
azureSubscription: '${{ parameters.azureSubscription }}'
ConnectedServiceName: '${{ parameters.appconnectionname}}'
appName: VALUE
ResourceGroupName: '${{ parameters.RG}}'
Essentially the first pipeline calls these two pipelines which are wrapped in Stages. Before they were not wrapped in stages and this pipeline worked well. Since moving it to stages I have had the issue where the task: DownloadPipelineArtifact#2 completes but downloads nothing. Please see screen shot bellow:
The error I am getting at the end of the pipeline is:
##[error]Error: No package found with specified pattern: /home/vsts/work/1/website/**/*.zip<br/>Check if the package mentioned in the task is published as an artifact in the build or a previous stage and downloaded in the current job.
I have tried the following solutions without success:
File pattern for Publish Pipeline Artifact in Azure DevOps
how to use PublishPipelineArtifact#1 with build script
And consulted MS Doc: https://learn.microsoft.com/en-us/azure/devops/pipelines/artifacts/pipeline-artifacts?view=azure-devops&tabs=yaml

How to fix 401 when deploying Maven build to Azure Artifacts via Azure Pipeline?

I've created feed in Azure Artifacts (azure-maven), added the MavenAuthenticate task the to build pipeline (with artifactsFeeds: azure-mave), added mavenAuthenticateFeed: true to the Maven task, added the repository to the pom.xml with the same ID, but when deploying the Maven task fails with 401 (Unauthorized).
Is there a step I have missed?
(I dont want to use a PAT if I dont have too, isnt that the reason to use MavenAuthenticate task?)
Cheers,
Steve
Hmmm, looks like the issue was actually setting mavenAuthenticateFeed to true, which doesnt make sense to me :(
Here is the working yml:
trigger:
- main
variables:
- name: MAVEN_CACHE_FOLDER
value: $(Pipeline.Workspace)/.m2/repository
- name: MAVEN_OPTS
value: -Dmaven.repo.local=$(MAVEN_CACHE_FOLDER)
pool:
vmImage: 'ubuntu-latest'
steps:
- task: MavenAuthenticate#0
inputs:
artifactsFeeds: 'azure-maven'
- task: Cache#2
inputs:
key: 'maven | "$(Agent.OS)" | pom.xml'
path: '$(MAVEN_CACHE_FOLDER)'
cacheHitVar: 'CacheRestored'
restoreKeys: |
maven | "$(Agent.OS)"
maven
displayName: Cache Maven local repo
- task: Maven#3
inputs:
mavenPomFile: 'pom.xml'
goals: 'package deploy'
publishJUnitResults: true
testResultsFiles: '**/surefire-reports/TEST-*.xml'
javaHomeOption: 'JDKVersion'
mavenVersionOption: 'Default'
mavenOptions: '-Xmx3072m $(MAVEN_OPTS)'
mavenAuthenticateFeed: false
jdkVersionOption: '1.8'
jdkArchitectureOption: 'x64'
effectivePomSkip: false
sonarQubeRunAnalysis: false
- task: CopyFiles#2
inputs:
Contents: '**/target/*.jar'
TargetFolder: '$(Build.ArtifactStagingDirectory)'
CleanTargetFolder: true
flattenFolders: true
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)'
ArtifactName: 'Test'
publishLocation: 'Container'
And if supplying your own settings.xml:
<servers>
<server>
<id>azure-maven</id>
<username>AzureDevOps</username>
<password>${env.SYSTEM_ACCESSTOKEN}</password>
</server>
</servers>
You need to set up the SYSTEM_ACCESSTOKEN:
- task: DownloadSecureFile#1
name: mvnSettings
inputs:
secureFile: 'settings.xml'
- task: Maven#3
env:
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
inputs:
mavenPomFile: 'pom.xml'
goals: 'clean deploy'
publishJUnitResults: true
testResultsFiles: '**/surefire-reports/TEST-*.xml'
javaHomeOption: 'JDKVersion'
mavenVersionOption: 'Default'
options: '-s $(mvnSettings.secureFilePath)'
mavenOptions: '-Xmx3072m $(MAVEN_OPTS)'
mavenAuthenticateFeed: false
jdkVersionOption: '1.8'
jdkArchitectureOption: 'x64'
effectivePomSkip: false
sonarQubeRunAnalysis: false

Azure starting 2 self hosted servers deliver files to Azure managed agent both in pool but only 1 accepting copyfiles task

I have a very strange behaver where I starting 2 self-hosted agents from my AWS server to deliver files to the managed agent in azure all with different names to be unique. but in the azure pipeline, i can see
both pipelines are taking the files from only 1 place instead of taking files from its unique place in the self-hosting server.
here is the command I'm starting the self-hosting docker server :
they all take form :
docker run -d --rm -t
--name=ios_docker_128_linux_slave_2
-e AZP_WORK=/var/jenkins/jenkins_slave/workspace/run_build_ios#2/128/ios_build_temp/working_dir
-v /var/jenkins/jenkins_slave/workspace/run_build_ios#2/128/ios_build_temp/working_dir:/azp
-e AZP_URL=https://dev.azure.com/xxx -e AZP_TOKEN=xxxx
-e AZP_AGENT_NAME=ios_docker_128_linux_slave_2
xxx.xxx.com:1/azure_self_hosted_agent/agent:latest
docker run -d --rm -t
--name=ios_docker_127_linux_slave_2
-e AZP_WORK=/var/jenkins/jenkins_slave/workspace/run_build_ios/127/ios_build_temp/working_dir
-v /var/jenkins/jenkins_slave/workspace/run_build_ios/127/ios_build_temp/working_dir:/azp
-e AZP_URL=https://dev.azure.com/xxx
-e AZP_TOKEN=xxxx
-e AZP_AGENT_NAME=ios_docker_127_linux_slave_2
xxx.xxx.com:1/azure_self_hosted_agent/agent:latest
but see in the azure pipeline they are taking the files fro the same place :
pay attention they all take the files from /run_build_ios/127
/var/jenkins/jenkins_slave/workspace/run_build_ios/127/ios_build_temp/working_dir/_temp/
look at the red markers
how my pipeline looks like :
pool:
vmImage: 'macOS 10.14'
parameters:
- name: Folderpath
type: string
displayName: 'configure path'
- name: FolderCompile
type: string
displayName: 'Compile ios productes path'
- name: projectName
type: string
displayName: 'ios projectName'
- name: appIdentifier
type: string
displayName: 'ios appIdentifier'
- name: versionNumber
type: string
displayName: 'ios versionNumber'
- name: buildNumber
type: string
displayName: 'ios buildNumber'
- name: plistFileFtpBasePath
type: string
displayName: 'ios plistFileFtpBasePath'
- name: fastlaneAppleSession
type: string
displayName: 'ios fastlaneAppleSession'
variables:
scheme: ''
sdk: 'iphoneos'
configuration: 'Release'
CERTIFICATE_PASSWORD: xxxx
FASTLANE_PASSWORD: xxxx
FASTLANE_SESSION: '${{parameters.fastlaneAppleSession}}'
jobs:
- job: self_hosted_connect
timeoutInMinutes: 10
pool: Default
steps:
- task: CopyFiles#2
inputs:
SourceFolder: '$(Agent.HomeDirectory)/../${{parameters.Folderpath}}'
Contents: '**'
TargetFolder: '$(build.artifactstagingdirectory)'
- task: PublishBuildArtifacts#1
inputs:
pathToPublish: '$(build.artifactstagingdirectory)'
artifactName: 'ios_artifacts'
- job: mac_agent
dependsOn: self_hosted_connect
timeoutInMinutes: 10
pool:
vmImage: 'macOS 10.14'
- task: UseRubyVersion#0
inputs:
versionSpec: '>= 2.4'
addToPath: true
- task: DownloadBuildArtifacts#0
inputs:
buildType: 'current'
downloadType: 'single'
artifactName: 'ios_artifacts'
downloadPath: '$(System.ArtifactsDirectory)'
- script: |
gem install --no-document bundler
bundle update --bundler
bundle install --retry=3 --jobs=4
gem install --no-document fastlane
mkdir fastlane
mv Fastfile fastlane
mv Appfile fastlane
pod deintegrate
gem install cocoapods
pod install
pod --version
fastlane release --verbose projectName:${{parameters.projectName}} appIdentifier:${{parameters.appIdentifier}} versionNumber:${{parameters.versionNumber}} buildNumber:${{parameters.buildNumber}} plistFileFtpBasePath:${{parameters.plistFileFtpBasePath}} ArtifactsDirectory:$(System.ArtifactsDirectory)
workingDirectory: '$(System.ArtifactsDirectory)/ios_artifacts'
displayName: 'create_keychain'
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)'
ArtifactName: 'Artifacts'
publishLocation: 'Container'
- job: copy_back_files_to_self_hosted_connect
dependsOn: mac_agent
timeoutInMinutes: 10
pool: Default
steps:
- task: DownloadBuildArtifacts#0
inputs:
buildType: 'current'
downloadType: 'single'
artifactName: 'Artifacts'
itemPattern: '**/*.ipa|manifest.plist'
downloadPath: '$(System.ArtifactsDirectory)'
- task: CopyFiles#2
inputs:
SourceFolder: '$(System.ArtifactsDirectory)'
Contents: '**/*.ipa|manifest.plist'
TargetFolder: '$(Agent.HomeDirectory)/../${{parameters.FolderCompile}}'
also here you can see i have 3 agents ( the name is a bit different as i took the picture later but it always 3 up )

Why my multi stage full yaml azure devops pipeline breaks when moved to template?

I have a multi stage full yaml pipeline in azure devops services. The different stages consist of a build stage, a Dev stage (Deploys azure resources to an azure resource group) and a UAT stage (Deploys azure resources to a different resource group). Everything works great. However I needed to use templates because I want to reuse the steps. After moving the steps to a template, the dev stage keeps working but the UAT stage gives authorization error. It is worth mentioning, dev and uat stages use different service connections targeting different subscriptions. But why will it work when I dont use templates and simply moving the steps to a template file all of a sudden it doesn't work. Is this a known bug?
This is the error I get when using the template approach:
| The client '9e5cc21a-bb38-46b9-a16a-289fbbf9c8b9' with object
| id '9e5cc21a-bb38-46b9-a16a-289fbbf9c8b9' does not have
| authorization to perform action
| 'Microsoft.Resources/subscriptions/resourceGroups/resources/read' over scope '/subscriptions/7041f5ba-1040-4989-8e48-497b3b826d01/resourceGroups/Resource-Group-Test-A' or the scope is invalid. If access was recently granted, please refresh your credentials. StatusCode: 403 ReasonPhrase: Forbidden OperationID : 464f577e-6617-4bed-9a14-1f7487b5f209
This is the pipeline without using templates (works perfect!).
# Starter pipeline
# Start with a minimal pipeline that you can customize to build and deploy your code.
# Add steps that build, run tests, deploy, and more:
# https://aka.ms/yaml
trigger:
- master
variables:
- group: 'Dev ARMT SFTP Connection'
resources:
repositories:
- repository: templates
type: git
name: CommonTasks
stages:
- stage: Build
jobs:
- job: Build
pool:
vmImage: 'ubuntu-latest'
steps:
- task: printAllVariables#1
- task: CopyFiles#2
inputs:
SourceFolder: '$(System.DefaultWorkingDirectory)'
Contents: 'ARM-Templates/**'
TargetFolder: '$(Build.ArtifactStagingDirectory)'
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)'
ArtifactName: 'drop'
publishLocation: 'Container'
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
# Write your PowerShell commands here.
Write-Host "Build Completed..."
$workingdir = "$(Build.ArtifactStagingDirectory)"
Write-Host $workingdir
$fcontent = Get-ChildItem -Path $workingdir
Write-Host $fcontent
- stage: Dev
jobs:
- job: Dev
pool:
vmImage: 'ubuntu-latest'
steps:
- task: DownloadBuildArtifacts#0
inputs:
buildType: 'specific'
project: '530cfd4b-51b8-4237-b2fa-f296a4cba29d'
pipeline: '30'
buildVersionToDownload: 'latest'
downloadType: 'single'
artifactName: 'drop'
downloadPath: '$(System.ArtifactsDirectory)/CommonTasks'
- task: DownloadBuildArtifacts#0
inputs:
buildType: 'current'
downloadType: 'single'
artifactName: 'drop'
downloadPath: '$(System.ArtifactsDirectory)/SftpConnection'
- task: FileTransform#1
displayName: 'ARMT SFTP Connection - Parameters File Transform '
inputs:
folderPath: '$(System.ArtifactsDirectory)/SftpConnection/drop'
fileType: json
targetFiles: 'ARM-Templates/parameters.json'
- task: AzurePowerShell#4
displayName: 'ARMT SFTP Connection - Check Resource Existance'
inputs:
azureSubscription: 'Starwood-DT-DEV-ServiceConnection'
ScriptPath: '$(System.ArtifactsDirectory)/CommonTasks/drop/AzurePowerShell/ResourceExistance.ps1'
ScriptArguments: '-resourceGroupName Starwood-DT-DEV -resourceName $(parameters.sftp_name.value)'
azurePowerShellVersion: LatestVersion
- task: AzureResourceGroupDeployment#2
displayName: 'ARMT SFTP Connection - Deploy'
inputs:
azureSubscription: 'Starwood-DT-DEV-ServiceConnection'
resourceGroupName: 'Starwood-DT-DEV'
location: 'East US'
csmFile: '$(System.ArtifactsDirectory)/SftpConnection/drop/ARM-Templates/template.json'
csmParametersFile: '$(System.ArtifactsDirectory)/SftpConnection/drop/ARM-Templates/parameters.json'
condition: eq(variables['deployresource'],'true')
- stage: GroupATest
jobs:
- job: GroupATest
pool:
vmImage: 'ubuntu-latest'
steps:
- task: DownloadBuildArtifacts#0
inputs:
buildType: 'specific'
project: '530cfd4b-51b8-4237-b2fa-f296a4cba29d'
pipeline: '30'
buildVersionToDownload: 'latest'
downloadType: 'single'
artifactName: 'drop'
downloadPath: '$(System.ArtifactsDirectory)/CommonTasks'
- task: DownloadBuildArtifacts#0
inputs:
buildType: 'current'
downloadType: 'single'
artifactName: 'drop'
downloadPath: '$(System.ArtifactsDirectory)/SftpConnection'
- task: FileTransform#1
displayName: 'ARMT SFTP Connection - Parameters File Transform '
inputs:
folderPath: '$(System.ArtifactsDirectory)/SftpConnection/drop'
fileType: json
targetFiles: 'ARM-Templates/parameters.json'
- task: AzurePowerShell#4
displayName: 'ARMT SFTP Connection - Check Resource Existance'
inputs:
azureSubscription: 'Resource-Group-Test-A'
ScriptPath: '$(System.ArtifactsDirectory)/CommonTasks/drop/AzurePowerShell/ResourceExistance.ps1'
ScriptArguments: '-resourceGroupName Resouce-Group-Test-A -resourceName $(parameters.sftp_name.value)'
azurePowerShellVersion: LatestVersion
- task: AzureResourceGroupDeployment#2
displayName: 'ARMT SFTP Connection - Deploy'
inputs:
azureSubscription: 'Resource-Group-Test-A'
resourceGroupName: 'Resouce-Group-Test-A'
location: 'East US'
csmFile: '$(System.ArtifactsDirectory)/SftpConnection/drop/ARM-Templates/template.json'
csmParametersFile: '$(System.ArtifactsDirectory)/SftpConnection/drop/ARM-Templates/parameters.json'
condition: eq(variables['deployresource'],'true')
When moving to steps template (then i get the error):
# Starter pipeline
# Start with a minimal pipeline that you can customize to build and deploy your code.
# Add steps that build, run tests, deploy, and more:
# https://aka.ms/yaml
trigger:
- master
resources:
repositories:
- repository: templates
type: git
name: CommonTasks
stages:
- stage: Build
jobs:
- job: Build
pool:
vmImage: 'ubuntu-latest'
steps:
- task: printAllVariables#1
- task: CopyFiles#2
inputs:
SourceFolder: '$(System.DefaultWorkingDirectory)'
Contents: 'ARM-Templates/**'
TargetFolder: '$(Build.ArtifactStagingDirectory)'
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)'
ArtifactName: 'drop'
publishLocation: 'Container'
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
# Write your PowerShell commands here.
Write-Host "Build Completed..."
$workingdir = "$(Build.ArtifactStagingDirectory)"
Write-Host $workingdir
$fcontent = Get-ChildItem -Path $workingdir
Write-Host $fcontent
- stage: Dev
variables:
- group: 'Dev ARMT SFTP Connection'
jobs:
- job: Dev
pool:
vmImage: 'ubuntu-latest'
steps:
- task: DownloadBuildArtifacts#0
inputs:
buildType: 'specific'
project: '530cfd4b-51b8-4237-b2fa-f296a4cba29d'
pipeline: '30'
buildVersionToDownload: 'latest'
downloadType: 'single'
artifactName: 'drop'
downloadPath: '$(System.ArtifactsDirectory)/CommonTasks'
- task: DownloadBuildArtifacts#0
inputs:
buildType: 'current'
downloadType: 'single'
artifactName: 'drop'
downloadPath: '$(System.ArtifactsDirectory)/SftpConnection'
- template: YamlTemplate/azure-resource-deploy.yml#templates
parameters:
dropLocation: '$(System.ArtifactsDirectory)/SftpConnection/drop'
transformTargetPath: 'ARM-Templates/parameters.json'
resourceName: $(parameters.sftp_name.value)
resourceGroupName: 'Starwood-DT-DEV'
azureServiceConnectionName: 'Starwood-DT-DEV-ServiceConnection'
resourceLocation: 'East US'
armtTemplateFilePath: '$(System.ArtifactsDirectory)/SftpConnection/drop/ARM-Templates/template.json'
armtParemeterFilePath: '$(System.ArtifactsDirectory)/SftpConnection/drop/ARM-Templates/parameters.json'
- stage: GroupATest
variables:
- group: 'GroupA ARMT SFTP Connection'
jobs:
- job: GroupATest
pool:
vmImage: 'ubuntu-latest'
steps:
- task: DownloadBuildArtifacts#0
inputs:
buildType: 'specific'
project: '530cfd4b-51b8-4237-b2fa-f296a4cba29d'
pipeline: '30'
buildVersionToDownload: 'latest'
downloadType: 'single'
artifactName: 'drop'
downloadPath: '$(System.ArtifactsDirectory)/CommonTasks'
- task: DownloadBuildArtifacts#0
inputs:
buildType: 'current'
downloadType: 'single'
artifactName: 'drop'
downloadPath: '$(System.ArtifactsDirectory)/SftpConnection'
- template: YamlTemplate/azure-resource-deploy.yml#templates
parameters:
dropLocation: '$(System.ArtifactsDirectory)/SftpConnection/drop'
transformTargetPath: 'ARM-Templates/parameters.json'
resourceName: $(parameters.sftp_name.value)
resourceGroupName: 'Resource-Group-Test-A'
azureServiceConnectionName: 'Resource-Group-Test-A'
resourceLocation: 'East US'
armtTemplateFilePath: '$(System.ArtifactsDirectory)/SftpConnection/drop/ARM-Templates/template.json'
armtParemeterFilePath: '$(System.ArtifactsDirectory)/SftpConnection/drop/ARM-Templates/parameters.json'
This the template structure
Parent template - (azure-resource-deploy.yml):
parameters:
- name: dropLocation
type: string
displayName: 'Artifact path to drop location'
default: ''
- name: transformTargetPath
type: string
default: ''
- name: resourceName
type: string
default: ''
- name: resourceGroupName
type: string
default: ''
- name: azureServiceConnectionName
type: string
default: ''
- name: resourceLocation
type: string
default: ''
- name: armtTemplateFilePath
type: string
default: ''
- name: armtParemeterFilePath
type: string
default: ''
steps:
- template: armt-parameter-file-transform.yml
parameters:
transformStepDisplayName: ''
folderPath: ${{parameters.dropLocation}}
targetFile: ${{parameters.transformTargetPath}}
- template: azure-resource-check.yml
parameters:
resourceName: ${{parameters.resourceName}}
resourceGroupName: ${{parameters.resourceGroupName}}
azureServiceConnectionName: ${{parameters.azureServiceConnectionName}}
- template: armt-deploy.yml
parameters:
resourceName: ${{parameters.resourceName}}
resourceGroupName: ${{parameters.resourceGroupName}}
resourceLocation: ${{parameters.resourceLocation}}
azureServiceConnectionName: ${{parameters.azureServiceConnectionName}}
templateFilePath: ${{parameters.armtTemplateFilePath}}
parametersFilePath: ${{parameters.armtParemeterFilePath}}
Templates referenced from parent:
(armt-parameter-file-transform.yml)
parameters:
- name: transformStepDisplayName
type: string
displayName: 'Display name for this step'
default: ''
- name: folderPath
type: string
displayName: 'Path to drop location'
default: ''
- name: targetFile
type: string
displayName: 'Path to paremeter file relative to drop location.'
default: ''
steps:
- task: FileTransform#1
displayName: ${{parameters.transformStepDisplayName}}
inputs:
folderPath: ${{parameters.folderPath}}
fileType: json
targetFiles: ${{parameters.targetFile}}
(azure-resource-check.yml)
parameters:
- name: resourceName
type: string
- name: resourceGroupName
type: string
- name: azureServiceConnectionName
type: string
steps:
- script: echo Echo -resourceGroupName ${{ parameters.resourceGroupName }} -resourceName ${{ parameters.resourceName }}
- task: AzurePowerShell#4
displayName: '${{ parameters.resourceName }} - Checking Resource Existance'
inputs:
azureSubscription: '${{ parameters.azureServiceConnectionName }}'
ScriptPath: '$(System.ArtifactsDirectory)/CommonTasks/drop/AzurePowerShell/ResourceExistance.ps1'
ScriptArguments: '-resourceGroupName ${{ parameters.resourceGroupName }} -resourceName ${{ parameters.resourceName }}'
azurePowerShellVersion: LatestVersion
(armt-deploy.yml)
parameters:
- name: resourceName
type: string
- name: resourceGroupName
type: string
- name: resourceLocation
type: string
- name: azureServiceConnectionName
type: string
- name: templateFilePath
type: string
- name: parametersFilePath
type: string
steps:
- task: AzureResourceGroupDeployment#2
displayName: 'ARMT Deploy - ${{parameters.resourceName}}'
inputs:
azureSubscription: ${{parameters.azureServiceConnectionName}}
resourceGroupName: ${{parameters.resourceGroupName}}
location: ${{parameters.resourceLocation}}
csmFile: ${{parameters.templateFilePath}}
csmParametersFile: ${{parameters.parametersFilePath}}
condition: eq(variables['deployresource'],'true')
This was just caused by a typo. The actual name of the resource group is Resouce-Group-Test-A, I missed the r when naming this resource group. The error does not happen in the non-templated version because when you pick the resource group you do it from a drop down list, with the correct prepopulated names, so no chance to make a mistake. However in a full yaml pipeline you have to type it in, and thats where the typo occured. If there is anything to be learned here. Pay close attention when typing resource names, errors might not be descriptive enough. Sorry If wasted anybodys precious time.

Resources