I'm new to azure. I was told that I need to make a condition: "if this is a request pool, then we run it depending on the content. If this is a build, then we run everything." As I see, each task already has a condition in all of this tasks:
condition: eq(dependencies.check.outputs['Check Changes.BE '], 'true')
As you can see, the task execution depends on the BE\FE variable. Task with that variables looks like this:
- task: CheckChanges#1
name: CheckChanges
pool: "windows-latest"
inputs:
rules: |
[FE]
**/*.json
**/*.ts
**/*.js
**/*.tsx
**/*.scss
**/*.html
**/*.css
[BE]
**/*.csproj
**/*.cs
**/*.yml
**/*.yaml
Here is the tasks with which it needs to be done:
- job: Restore BE
displayName: Restore BE
dependsOn: CheckChanges
condition: eq(dependencies.check.outputs['CheckChanges.BE'], 'true')
steps:
- task: Cake#2
displayName: Restore FE and Sitecore modules
inputs:
script: "$(Build.Repository.LocalPath)/src/build.cake"
target: "Server :: Restore"
verbosity: "Quiet"
Version: "1.3.0"
- job: Restore FE
displayName: Restore FE
dependsOn: CheckChanges
condition: eq(dependencies.check.outputs['CheckChanges.FE'], 'true')
steps:
- task: Cake#2
displayName: Restore FE and Sitecore modules
inputs:
script: "$(Build.Repository.LocalPath)/src/build.cake"
target: "Client :: Restore"
verbosity: "Quiet"
Version: "1.3.0"
- job: Build BE
displayName: Build BE
dependsOn: CheckChanges
condition: eq(dependencies.check.outputs['CheckChanges.BE'], 'true')
steps:
- task: Cake#2
displayName: Generate and Build FE and Sitecore
inputs:
script: "$(Build.Repository.LocalPath)/src/build.cake"
target: "Server :: Build"
verbosity: "Quiet"
arguments: '--BuildConfiguration "Release" --ScSiteUrl "dummy"'
Version: "1.3.0"
- job: Build FE
displayName: Build FE
dependsOn: CheckChanges
condition: eq(dependencies.check.outputs['CheckChanges.FE'], 'true')
steps:
- task: Cake#2
displayName: Generate and Build FE and Sitecore
inputs:
script: "$(Build.Repository.LocalPath)/src/build.cake"
target: "Client :: Build"
verbosity: "Quiet"
arguments: '--BuildConfiguration "Release" --ScSiteUrl "dummy"'
Version: "1.3.0"
- job: Unit tests BE
displayName: Unit tests BE
dependsOn: CheckChanges
condition: eq(dependencies.check.outputs['CheckChanges.BE'], 'true')
steps:
- task: Cake#2
displayName: Run Unit tests
inputs:
script: "$(Build.Repository.LocalPath)/src/build.cake"
target: "Server :: Tests"
verbosity: "Quiet"
Version: "1.3.0"
- job: Unit tests FE
displayName: Unit tests FE
dependsOn: CheckChanges
condition: eq(dependencies.check.outputs['CheckChanges.FE'], 'true')
steps:
- task: Cake#2
displayName: Run Unit tests
inputs:
script: "$(Build.Repository.LocalPath)/src/build.cake"
target: "Client :: Tests"
verbosity: "Quiet"
Version: "1.3.0"
I don't understand what I need to do. My Team Lead just say: "If it's a Pull Request, then we run it depending on the content. If it's a build, then we run everything."
If it's a Pull Request, then we run it depending on the content. If it's a build, then we run everything.
You can use the Pipeline Predefined variable: Build.Reason to confirm if the pipeline is triggered by Pull Request or other method.
You can add the predefined variable in condition and use expression to link the filters.
For example:
Add condition to CheckChanges task.
- task: CheckChanges#1
condition: eq(variables['Build.Reason'], 'PullRequest')
Add condition to other jobs:
- job: Restore BE
displayName: Restore BE
dependsOn: CheckChanges
condition: Or(eq(dependencies.check.outputs['CheckChanges.FE'], 'true'),ne(variables['Build.Reason'], 'PullRequest'))
steps:
- task: Cake#2
displayName: Restore FE and Sitecore modules
inputs:
script: "$(Build.Repository.LocalPath)/src/build.cake"
target: "Server :: Restore"
verbosity: "Quiet"
Version: "1.3.0"
Explanation:
When the Pipeline is triggered by Pull Request, the checkChanges task will run and the tasks will run based on the content.
Or the pipeline will run all tasks, if the pipeline is not triggered by Pull Request.
For more detailed info, you can refer to the doc: Condition, Expression, Predefined Variable
Related
My YAML pipeline has 3 deployment stages:
Alpha servers -> Beta servers -> Production servers
I have 3 branches. One for alpha code, one for beta, and one for production. What I want to achieve is to automate the following workflow in YAML(I've already set a trigger for the alpha branch, but it's a question about later stages):
Changes merged to the alpha branch after a pull request is completed -> alpha branch code build -> deployment to alpha servers -> *manual testing performed on alpha servers* -> only these specific changes merged from alpha to beta branch -> beta branch code build -> deployment to beta servers -> *manual testing performed on beta servers* -> only these specific changes merged from beta to main branch -> main branch code build -> deployment to production servers
I can't figure out how to make beta and production stages run when changes are merged to the beta and main branches. If I declare triggers for all 3 branches (alpha, beta, main) it will create a new pipeline run every time so instead of having 1 run with specific changes:
alpha(run after merge to alpha branch) -> beta(run after merge to beta branch) -> prod(run after merge to main branch)
I'm using conditions for alpha, beta, prod deployments to check which branch has been merged to but I end up with 3 runs:
alpha(run after merge to alpha branch) -> beta(did not run) -> prod(did not run)
alpha(did not run) -> beta(run after merge to beta branch) -> prod(did not run)
alpha(did not run) -> beta(did not run) -> prod(run after merge to main branch)
My pipeline:
variables:
- name: app_name
value: 'myapp'
trigger:
batch: true
branches:
include:
- alpha
- beta
stages:
- stage: buildAlpha
condition: and(eq(variables['Build.Reason'], 'IndividualCI'), eq(variables['Build.SourceBranch'], 'refs/heads/alpha'))
pool:
vmImage: windows-2022
jobs:
- job: compile
steps:
- checkout: git://MyTeam/MyProjectTest#alpha
- task: UseDotNet#2
displayName: 'Use .NET Core 3.1'
inputs:
packageType: 'sdk'
version: '3.1.x'
- task: NuGetToolInstaller#0
displayName: 'Use NuGet 6.0.0'
inputs:
versionSpec: 6.0.0
- task: NuGetCommand#2
displayName: 'NuGet restore'
inputs:
restoreSolution: 'MyProject\MyProject.sln'
- task: DotNetCoreCLI#2
inputs:
command: 'publish'
publishWebProjects: false
projects: 'MyProject\MyProject.sln'
arguments: '-o $(build.artifactstagingdirectory)'
modifyOutputPath: false
zipAfterPublish: true
- publish: '$(build.artifactstagingdirectory)'
artifact: drop
- stage: deployDev
condition: and(eq(variables['Build.Reason'], 'IndividualCI'), eq(variables['Build.SourceBranch'], 'refs/heads/alpha'))
jobs:
- deployment: deployToAlpha
variables:
- group: DevVariables
environment:
name: dev
resourceType: VirtualMachine
tags: web
strategy:
runOnce:
deploy:
steps:
- task: IISWebAppDeploymentOnMachineGroup#0
displayName: 'IIS Web App Deploy'
inputs:
WebSiteName: 'myapp.og.com'
VirtualApplication: $(app_name)
RemoveAdditionalFilesFlag: true
TakeAppOfflineFlag: True
XmlVariableSubstitution: True
Package: '$(Pipeline.Workspace)/drop/**/*.zip'
- task: qetza.replacetokens.replacetokens-task.replacetokens#4
displayName: 'Replace tokens in **/appsettings.json'
inputs:
targetFiles: 'F:/Websites/myapp.og.com/$(app_name)/appsettings.json'
actionOnMissing: fail
keepToken: true
actionOnNoFiles: fail
enableTelemetry: false
- stage: buildBeta
condition: and(eq(variables['Build.Reason'], 'IndividualCI'), eq(variables['Build.SourceBranch'], 'refs/heads/beta'))
pool:
vmImage: windows-2022
jobs:
- job: compile
steps:
- checkout: git://MyTeam/MyProjectTest#beta
- task: UseDotNet#2
displayName: 'Use .NET Core 3.1'
inputs:
packageType: 'sdk'
version: '3.1.x'
- task: NuGetToolInstaller#0
displayName: 'Use NuGet 6.0.0'
inputs:
versionSpec: 6.0.0
- task: NuGetCommand#2
displayName: 'NuGet restore'
inputs:
restoreSolution: 'MyProject\MyProject.sln'
- task: DotNetCoreCLI#2
inputs:
command: 'publish'
publishWebProjects: false
projects: 'MyProject\MyProject.sln'
arguments: '-o $(build.artifactstagingdirectory)'
modifyOutputPath: false
zipAfterPublish: true
- publish: '$(build.artifactstagingdirectory)'
artifact: drop
- stage: deployBeta
condition: and(eq(variables['Build.Reason'], 'IndividualCI'), eq(variables['Build.SourceBranch'], 'refs/heads/beta'))
jobs:
- deployment: deployToBeta
variables:
- group: betavariables
environment:
name: beta
resourceType: VirtualMachine
tags: web
strategy:
runOnce:
deploy:
steps:
- task: IISWebAppDeploymentOnMachineGroup#0
displayName: 'IIS Web App Deploy'
inputs:
WebSiteName: 'myapp.og.com'
VirtualApplication: $(app_name)
RemoveAdditionalFilesFlag: true
TakeAppOfflineFlag: True
XmlVariableSubstitution: True
Package: '$(Pipeline.Workspace)/drop/**/*.zip'
- task: qetza.replacetokens.replacetokens-task.replacetokens#4
displayName: 'Replace tokens in **/appsettings.json'
inputs:
targetFiles: 'F:/Websites/myapp.og.com/$(app_name)/appsettings.json'
actionOnMissing: fail
keepToken: true
actionOnNoFiles: fail
enableTelemetry: false
In your scenario, Pull Request merges in three branches trigger three pipeline runs. This is an expected behavior.
In Yaml Pipeline, the trigger can only be set at the root level. It works on the entire pipeline. So three PR merges will trigger three pipeline runs, and they are independent。
When PR merge triggers the pipeline, it will run the matching stage based on the condition.
Variables will be evaluated with the same value, so some stages will run and others will skip.
For your requirement: three PR merges trigger different stages of the same pipeline run , I am afraid that there is no out-of-box method can meet your requirement.
For a workaround, you can split into three pipelines according to the stages and set the corresponding trigger for each pipeline.
I suggest that you can create a suggestion feedback to report the feature requirement.
I have a react app, created with npx create-react-app command, and configured a yaml based azure devops pipeline for CI/CD. Then I added some simple unit tests.
I added a script task in my yaml pipeline to generate coverage results as follows.
- script: |
cd reactonazure
npm run test -- --coverage --ci --reporters=default --reporters=jest-junit --coverageReporters=cobertura
displayName: 'test code coverage'
But this tasks displays the following output and then task continues to run and never ends. I have to cancel the pipeline run to ultimately stop it. Not sure whats happening. Any ideas?
No tests found related to files changed since last commit.
But when I prefix with CI=true to make the command as follows, then the command works fine and I get test results, but no where I find coverage report(cobertura-coverage.xml).
CI=true npm test -- --reporters=jest-junit --reporters=default --coverageReporters=cobertura
I added a simple script task shown below to list out the files and folders to enable me search for cobertura-coverage.xml. But no where I found the file. Note that jest.xml file is being created. And that is the reason why I am able to see the test results shown above. The problem is with test coverage report.
- script: |
pwd
cd ..
pwd
ls -Rla
displayName: 'list out the files and folders for inspection'
Also when I run the following command on vs code command prompt on my machine(Windows 10), I do see a file named cobertura-coverage.xml in coverage folder.
npm run test -- --coverage --ci --reporters=default --reporters=jest-junit --coverageReporters=cobertura
The full pipeline yaml is pasted below.
trigger:
- master
variables:
azureSubscription: 'AzureServiceConnection'
# Web app name
webAppName: BasicReactApp
# Environment name
environmentName: Dev
# Agent VM image name
vmImageName: 'ubuntu-latest'
stages:
- stage: Build
displayName: Build stage
jobs:
- job: Build
displayName: Build
pool:
vmImage: $(vmImageName)
steps:
- task: NodeTool#0
inputs:
versionSpec: '16.x'
displayName: 'Install Node.js'
- script: |
cd reactonazure
npm install
npm run build --if-present
displayName: 'npm install, build '
- script: |
cd reactonazure
CI=true npm test -- --reporters=jest-junit --reporters=default --coverageReporters=cobertura
displayName: 'test code coverage'
- task: PublishTestResults#2
displayName: "Publish Test Results"
inputs:
testResultsFormat: 'JUnit'
testResultsFiles: 'reactonazure/junit.xml'
failTaskOnFailedTests: true
mergeTestResults: true
condition: succeededOrFailed()
# The following, two are not functioning as no coverage.cobertura.xml file is being created.
- task: PublishCodeCoverageResults#1
displayName: "Publish code coverage"
inputs:
codeCoverageTool: Cobertura
summaryFileLocation: "reactonazure/$(System.DefaultWorkingDirectory)/coverage/cobertura-coverage.xml"
reportDirectory: "reactonazure/$(System.DefaultWorkingDirectory)/coverage"
failIfCoverageEmpty: false
- task: PublishCodeCoverageResults#1
displayName: 'Publish code coverage report'
inputs:
codeCoverageTool: 'Cobertura'
summaryFileLocation: '$(Agent.TempDirectory)/**/coverage.cobertura.xml'
failIfCoverageEmpty: true
Are there any other ways to generate report on the pipeline?
The path you provided to the coverage file is incorrect. It should be $(System.DefaultWorkingDirectory)/reactonazure/coverage/cobertura-coverage.xml.
The task would look like this:
- task: PublishCodeCoverageResults#1
displayName: "Publish code coverage"
inputs:
codeCoverageTool: Cobertura
summaryFileLocation: "$(System.DefaultWorkingDirectory)/reactonazure/coverage/cobertura-coverage.xml"
reportDirectory: "$(System.DefaultWorkingDirectory)/reactonazure/coverage"
failIfCoverageEmpty: false
The command to generate the report that I presented earlier is not working. The following is finally working.
CI=true npm run test -- --coverage --watchAll=false --ci --reporters=default --reporters=jest-junit --coverageReporters=cobertura
The full and final yaml after taking into account the suggested answer is below.
trigger:
- master
variables:
azureSubscription: 'AzureServiceConnection'
webAppName: BasicReactApp
environmentName: Dev
vmImageName: 'ubuntu-latest'
stages:
- stage: Build
displayName: Build stage
jobs:
- job: Build
displayName: Build
pool:
vmImage: $(vmImageName)
steps:
- task: NodeTool#0
inputs:
versionSpec: '16.x'
displayName: 'Install Node.js'
- script: |
cd reactonazure
npm install
npm run build --if-present
displayName: 'npm install, build '
- script: |
cd reactonazure
CI=true npm run test -- --coverage --watchAll=false --ci --reporters=default --reporters=jest-junit --coverageReporters=cobertura
displayName: 'Test with code coverage'
- task: PublishTestResults#2
displayName: "Publish Test Results"
inputs:
testResultsFormat: 'JUnit'
testResultsFiles: 'reactonazure/junit.xml'
failTaskOnFailedTests: true
mergeTestResults: true
condition: succeededOrFailed()
- task: PublishCodeCoverageResults#1
displayName: "Publish code coverage"
inputs:
codeCoverageTool: Cobertura
summaryFileLocation: "$(System.DefaultWorkingDirectory)/reactonazure/coverage/cobertura-coverage.xml"
reportDirectory: "$(System.DefaultWorkingDirectory)/reactonazure/coverage"
failIfCoverageEmpty: false
- task: ArchiveFiles#2
displayName: 'Archive files'
inputs:
rootFolderOrFile: '$(System.DefaultWorkingDirectory)/reactonazure/build'
includeRootFolder: false
archiveType: zip
archiveFile: $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
replaceExistingArchive: true
- upload: $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
artifact: drop
- stage: Deploy
displayName: Deploy stage
dependsOn: Build
condition: succeeded()
jobs:
- deployment: Deploy
displayName: Deploy
environment: $(environmentName)
pool:
vmImage: $(vmImageName)
strategy:
runOnce:
deploy:
steps:
- task: AzureWebApp#1
displayName: 'Azure Web App Deploy'
inputs:
azureSubscription: $(azureSubscription)
#appType: webAppLinux
appType: webApp
appName: $(webAppName)
runtimeStack: 'NODE|16.x'
package: $(Pipeline.Workspace)/drop/$(Build.BuildId).zip
startUpCommand: 'npm run start'
I have implemented a pipeline that will build and test the application once a Pull Request have been created. It works fine but it will run again once the PR is merged, and I only want it to run in the creation of a PR.
The Yaml fine goes like:
trigger:
- development
pool:
vmImage: ubuntu-latest
variables:
buildConfiguration: 'Release'
steps:
- task: UseDotNet#2
displayName: 'Use .NET Core sdk 3.1.000'
inputs:
version: 3.1.x
- task: DotNetCoreCLI#2
displayName: 'Build the application'
inputs:
command: 'build'
projects: 'example.csproj'
configuration: 'Release'
- task: DotNetCoreCLI#2
displayName: 'Run unit tests'
inputs:
command: 'test'
projects: 'example.csproj'
configuration: 'Release'
kraego is right.
You will have to remove the trigger section within your pipeline and configure a build validation branch policy for that build.
See here.
Im having real issues with a pipeline everytime someone commits or pushes something to a branch on our repo, the pipeline triggers, in following the Microsoft Doc: https://learn.microsoft.com/en-us/azure/devops/pipelines/repos/azure-repos-git?view=azure-devops&tabs=yaml#ci-triggers
Putting in Exclude features on every branch that we have the pipeline will still run when someone does a commit to a local branch even if I have wild carded the branch.
Has anyone been able to get this to work, that the pipeline should only run when there is a commit to Master only and nothing else.
Here is my Code:
trigger:
branches:
include:
- master
exclude:
- CICV/*
- An/*
- Prod/*
- Test/*
- Dev/*
- dev/*
- IN/*
- id/*
- St/*
- tr/*
pool:
vmImage: 'windows-latest'
demands: npm
variables:
System.Debug: false
azureSubscription: 'RunPipelinesInProd'
RG: 'VALUE'
Location: UK South
containername: 'private'
appconnectionname: 'RunPipelinesInProd'
jobs:
- job: job1
displayName: Create And Publish Artifact
pool:
vmImage: vs2017-win2016
steps:
- task: UseDotNet#2
displayName: Use .Net Core 3.1.x SDK
inputs:
packageType: 'sdk'
version: '3.1.x'
- task: DotNetCoreCLI#2
displayName: dotnet restore
inputs:
command: restore
projects: 'Website.csproj'
- task: Npm#1
displayName: 'npm install'
inputs:
workingDir: ClientApp
verbose: false
- task: Npm#1
displayName: 'npm run build'
inputs:
command: 'custom'
workingDir: ClientApp
customCommand: 'build'
- task: DotNetCoreCLI#2
displayName: dotnet build
inputs:
projects: 'Website.csproj'
arguments: '--configuration Release'
- task: DotNetCoreCLI#2
displayName: dotnet Test
inputs:
command: test
projects: 'UnitTests/UnitTests.csproj'
arguments: '--configuration Release'
- task: DotNetCoreCLI#2
displayName: dotnet publish
inputs:
command: publish
projects: 'Website.csproj'
arguments: '--configuration Release --output $(Build.ArtifactStagingDirectory)'
zipAfterPublish: true
modifyOutputPath: false
- task: PublishPipelineArtifact#1
displayName: Publish Pipeline Artifact
inputs:
targetPath: '$(Build.ArtifactStagingDirectory)'
artifact: 'Website'
publishLocation: 'pipeline'
- job: job2
displayName: Create Web App
dependsOn: job1
steps:
# Download Artifact File
- download: none
- task: DownloadPipelineArtifact#2
displayName: 'Download Build Artifacts'
inputs:
patterns: '**/*.zip'
path: '$(Build.ArtifactStagingDirectory)'
# deploy to Azure Web App
- task: AzureWebApp#1
displayName: 'Azure Web App Deploy: nsclassroom-dgyn27h2dfoyo'
inputs:
package: $(Build.ArtifactStagingDirectory)/**/*.zip
azureSubscription: $(azureSubscription)
ConnectedServiceName: $(appconnectionname)
appName: 'VALUE'
ResourceGroupName: $(RG)
You don't need a complex trigger like the one you outlined to trigger the pipeline on pushes to master. The following simple trigger configuration should work:
trigger:
- master
If there's anything in the include section, then only pushes to these branches trigger the build. If you specify both include and exclude sections, then it will try to exclude some subset from the include set - just like in the sample from the docs:
# specific branch build
trigger:
branches:
include:
- master
- releases/*
exclude:
- releases/old*
If the pipeline is still triggered by the push to some other branch, then it must be something else that triggers it.
As mentioned by #yan-sklyraneko in this answer your trigger configuration should be as simple as
trigger:
- master
However the trigger in your YAML file can be overridden in the GUI. Navigate to your pipeline and click Edit then click the ellipses as shown below and select Triggers
On that screen check that the Override the YAML continuous integration trigger from here box is unticked
I solved this in the end, I ended up down the route of managing through the Azure Dev Ops Portal.
It seems that if you try to use YAML to manage this it just doesn't work, but if you do it through the web interface as outlined in Answer 2, then the behaviour is as expected. I think that the Microsoft YAML part is broken for this but I already have three issues open with Microsoft I don't wish to add another one to follow and tag.
I´m struggling to make my MultiStage pipelines to run a .exe file in a hosted agent running in a Azure VM.
My .yaml file is:
trigger:
- develop
stages:
- stage: build
displayName: Build
jobs:
- job: buildJob
pool:
vmImage: 'ubuntu-16.04'
variables:
buildConfiguration: 'Release'
steps:
- task: NuGetToolInstaller#1
inputs:
versionSpec: '5.5.0'
- task: DotNetCoreCLI#2
displayName: 'Dotnet Build $(buildConfiguration)'
inputs:
command: 'build'
arguments: '--configuration $(buildConfiguration)'
projects: '**/TestProj.csproj'
- task: DotNetCoreCLI#2
displayName: "Publish"
inputs:
command: 'publish'
publishWebProjects: false
projects: '**/TestProj.csproj'
arguments: '--no-restore --configuration $(BuildConfiguration) --output $(Build.ArtifactStagingDirectory)'
zipAfterPublish: false
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)'
ArtifactName: 'drop'
publishLocation: Container
- stage: Release
displayName: Release
dependsOn: build
jobs:
- deployment: AzureVMDeploy
displayName: agentDeploy
environment:
name: AzureDeploy
resourceName: vmName
resourceType: VirtualMachine
tags: develop
This VM is on the azure pipelines Environment. After I run this pipeline, the folder is downloaded into the VM, but I cannot find how to automate the execution of the output .exe file in this folder.
I think the way is to create a job with a task to do it, but I cannot figure out how to set the agent installed on the VM to run this task.
How can I do that?
If I understood you correctly, you want to execute your artifact file which was deployed to VM.
I think that PowerShell on Target Machines task should do the job for you. Yoy can write simple inline script to exeute your file. However, you need to have remoting confogured on VM. This article may help you with this.
You could specify tasks in strategy: Deployment job For example:
YAML
stages:
- stage: build
jobs:
- job: buildJob
pool:
vmImage: 'Ubuntu-16.04'
steps:
- task: PublishPipelineArtifact#1
inputs:
targetPath: '$(Pipeline.Workspace)'
publishLocation: 'pipeline'
- stage: deploy
dependsOn: build
jobs:
- deployment: DeployWeb
displayName: deploy Web App
environment:
name: vm1
resourceType: virtualmachine
strategy:
runOnce:
deploy:
steps:
- script: echo my first deployment
- task: CmdLine#2
inputs:
script: 'more README.md'
workingDirectory: '$(Pipeline.Workspace)/build.buildJob/s'
For this YAML pipeline, I publish all files in pipeline workspace to artifact in build stage, then this artifact will be download to target virtual machine of vm1 environment in deploy stage (folder name will be {stage name}.{job name}), then run command line task to get a file's content. (The script and command line tasks will be run on that virtual machine)