Does Azure DevOps Pipelines cache some data accross runs - azure

I'm new to Azure DevOps and pipelines, and I ran into an issue running the same pipeline multiple times in a short period.
In brief, I created a pipeline to simply build a .Net project with MSBuild and generate an artifact. The pipeline trigger on change in master branch.
The first time, it worked, I can download the artifact and execute the program without any issue. Now if I do a change in the master branch 5 minutes later adding an option to my program, the pipeline runs successfully, however when running program stored in the generated artifact, my new option is not there.
I'm probably doing something stupid there, but I don't understand why I have this behaviour.
Is there any kind of caching and how can I have fresh build everytime ?
== EDIT ==
Here is my YAML definition as requested
Basically, steps are:
Checkout solution with all submodule
Nuget restore packages for all required projects
MSBuild task
Archive the output
Publish artifact.
trigger:
- master
pool:
demands: azureps
vmImage: 'windows-latest'
steps:
- checkout: "git://GSS-CMDB-Tools/GSSAM_Code"
submodules: true
persistCredentials: true
- task: NuGetCommand#2
inputs:
command: 'custom'
arguments: 'restore ADDMSync/packages.config -SolutionDirectory .'
- task: NuGetCommand#2
inputs:
command: 'custom'
arguments: 'restore GSSAM/packages.config -SolutionDirectory .'
- task: NuGetCommand#2
inputs:
command: 'custom'
arguments: 'restore GSSAM.ADDMRest/packages.config -SolutionDirectory .'
- task: NuGetCommand#2
inputs:
command: 'custom'
arguments: 'restore GSSAM.SNOWRest/packages.config -SolutionDirectory .'
- task: MSBuild#1
inputs:
solution: 'ADDMSync/ADDMSync.csproj'
msbuildArchitecture: 'x64'
configuration: 'Release'
msbuildArguments: '/p:PostBuildEvent='
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
# Write your PowerShell commands here.
mv ADDMSync/bin/Release ADDMSync/Bin/ADDMSync
rm ADDMSync/bin/ADDMSync/*.pdb
- task: ArchiveFiles#2
inputs:
rootFolderOrFile: 'ADDMSync/bin/ADDMSync'
includeRootFolder: true
archiveType: 'zip'
archiveFile: '$(Build.ArtifactStagingDirectory)/ADDMSync.zip'
replaceExistingArchive: true
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)/ADDMSync.zip'
ArtifactName: 'ADDMSync'
publishLocation: 'Container'
Thanks a lot
RĂ©mi

OK I think I understand what happens.
What I did was to commit and push all submodules required by the build. However I did not commit the modification of the solution itself. By doing so it makes it working.
I don't understand why for now, I guess it's link to the way the checkout task works.

Related

Azure pipeline selenium save screenshot

I have an azure pipeline that triggers a python selenium scrip that check that my website works properly.
But there is a stage that keeps failing because I need selenium to input a specific date, and as I am not sure if the date inputed is in the wrong format (locally works just fine) I would like to take a screenshot at that stage to fully understand what is happening in there.
locally this is my configuration to save the screenshot:
try:
wait.until(EC.element_to_be_clickable((By.XPATH, '//*[#id="root"]/div[2]/main/div[4]/div/button[2]'))).click()
except:
driver.save_screenshot('error.png')
This works just fine and it does output the png image in the local folder.
but running this on azure pipeline, is not saving the png file.
this is my pipeline configuration
stages:
- stage:
jobs:
- job: Configuration
steps:
- task: UsePythonVersion#0
inputs:
versionSpec: '3.8'
addToPath: true
- script: |
python -m pip install --upgrade pip
pip install selenium
printenv
- task: Pythonscript#0
inputs:
scriptSource: 'filePath'
scriptPath: './script1.py'
env:
USERNAMEOT: $(usernameot)
PASSWORDOT: $(passwordot)
- job: Artifact
steps:
- task: CopyFiles#2
displayName: 'Copy Files to: $(build.artifactstagingdirectory)'
inputs:
SourceFolder: '$(system.defaultworkingdirectory)'
Contents: '**.png'
TargetFolder: '$(build.artifactstagingdirectory)'
flattenFolders: true
- task: PublishBuildArtifacts#1
displayName: 'Publish Artifact: screenshots'
inputs:
PathtoPublish: '$(build.artifactstagingdirectory)'
ArtifactName: screenshots
I do have a task to copy file and publish the artefact, but as this task runs in parallel, it completes before the previous job and return nothing.
I was wandering how can I save the png file I have to the artefact folder even if the Configuration job fails?
Thank you so much for any help you can provide me guys, I am really struggling on this
I was wandering how can I save the png file I have to the artefact folder even if the Configuration job fails?
You could try to set the Dependencies and condition for the job Artifact, like:
jobs:
- job: Artifact
dependsOn: Configuration
condition: succeededOrFailed()
With the dependsOn: Configuration, the job Artifact will executed after the job Configuration. And the condition: succeededOrFailed() will keep the job Artifact execute when the Configuration job fails.
You could check the document Specify conditions and Dependencies for some more details.

CI Triggers on Pipelines in Azure

Im having real issues with a pipeline everytime someone commits or pushes something to a branch on our repo, the pipeline triggers, in following the Microsoft Doc: https://learn.microsoft.com/en-us/azure/devops/pipelines/repos/azure-repos-git?view=azure-devops&tabs=yaml#ci-triggers
Putting in Exclude features on every branch that we have the pipeline will still run when someone does a commit to a local branch even if I have wild carded the branch.
Has anyone been able to get this to work, that the pipeline should only run when there is a commit to Master only and nothing else.
Here is my Code:
trigger:
branches:
include:
- master
exclude:
- CICV/*
- An/*
- Prod/*
- Test/*
- Dev/*
- dev/*
- IN/*
- id/*
- St/*
- tr/*
pool:
vmImage: 'windows-latest'
demands: npm
variables:
System.Debug: false
azureSubscription: 'RunPipelinesInProd'
RG: 'VALUE'
Location: UK South
containername: 'private'
appconnectionname: 'RunPipelinesInProd'
jobs:
- job: job1
displayName: Create And Publish Artifact
pool:
vmImage: vs2017-win2016
steps:
- task: UseDotNet#2
displayName: Use .Net Core 3.1.x SDK
inputs:
packageType: 'sdk'
version: '3.1.x'
- task: DotNetCoreCLI#2
displayName: dotnet restore
inputs:
command: restore
projects: 'Website.csproj'
- task: Npm#1
displayName: 'npm install'
inputs:
workingDir: ClientApp
verbose: false
- task: Npm#1
displayName: 'npm run build'
inputs:
command: 'custom'
workingDir: ClientApp
customCommand: 'build'
- task: DotNetCoreCLI#2
displayName: dotnet build
inputs:
projects: 'Website.csproj'
arguments: '--configuration Release'
- task: DotNetCoreCLI#2
displayName: dotnet Test
inputs:
command: test
projects: 'UnitTests/UnitTests.csproj'
arguments: '--configuration Release'
- task: DotNetCoreCLI#2
displayName: dotnet publish
inputs:
command: publish
projects: 'Website.csproj'
arguments: '--configuration Release --output $(Build.ArtifactStagingDirectory)'
zipAfterPublish: true
modifyOutputPath: false
- task: PublishPipelineArtifact#1
displayName: Publish Pipeline Artifact
inputs:
targetPath: '$(Build.ArtifactStagingDirectory)'
artifact: 'Website'
publishLocation: 'pipeline'
- job: job2
displayName: Create Web App
dependsOn: job1
steps:
# Download Artifact File
- download: none
- task: DownloadPipelineArtifact#2
displayName: 'Download Build Artifacts'
inputs:
patterns: '**/*.zip'
path: '$(Build.ArtifactStagingDirectory)'
# deploy to Azure Web App
- task: AzureWebApp#1
displayName: 'Azure Web App Deploy: nsclassroom-dgyn27h2dfoyo'
inputs:
package: $(Build.ArtifactStagingDirectory)/**/*.zip
azureSubscription: $(azureSubscription)
ConnectedServiceName: $(appconnectionname)
appName: 'VALUE'
ResourceGroupName: $(RG)
You don't need a complex trigger like the one you outlined to trigger the pipeline on pushes to master. The following simple trigger configuration should work:
trigger:
- master
If there's anything in the include section, then only pushes to these branches trigger the build. If you specify both include and exclude sections, then it will try to exclude some subset from the include set - just like in the sample from the docs:
# specific branch build
trigger:
branches:
include:
- master
- releases/*
exclude:
- releases/old*
If the pipeline is still triggered by the push to some other branch, then it must be something else that triggers it.
As mentioned by #yan-sklyraneko in this answer your trigger configuration should be as simple as
trigger:
- master
However the trigger in your YAML file can be overridden in the GUI. Navigate to your pipeline and click Edit then click the ellipses as shown below and select Triggers
On that screen check that the Override the YAML continuous integration trigger from here box is unticked
I solved this in the end, I ended up down the route of managing through the Azure Dev Ops Portal.
It seems that if you try to use YAML to manage this it just doesn't work, but if you do it through the web interface as outlined in Answer 2, then the behaviour is as expected. I think that the Microsoft YAML part is broken for this but I already have three issues open with Microsoft I don't wish to add another one to follow and tag.

azure devops build and deploy to app service

I have create an empty project on dev.azure.com
I have cloned the repository on my local computer.
I have run this command in the main folder:
$ dotnet new mvc
I have create this azure-pipelines.yml file:
trigger:
- master
pool:
vmImage: 'windows-latest'
steps:
- task: DotNetCoreCLI#2
inputs:
command: 'restore'
feedsToUse: 'select'
- task: DotNetCoreCLI#2
inputs:
command: 'build'
- task: DotNetCoreCLI#2
inputs:
command: 'publish'
publishWebProjects: true
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)'
ArtifactName: 'artifact2'
I have add, commit and pushed files on dev.azure.com (on master branch)
I have this warning message:
##[warning]Directory 'd:\a\1\a' is empty. Nothing will be added to build artifact 'artifact2'.
I have create a release pipeline but i get an error:
##[error]Error: No package found with specified pattern: D:\a\r1\a\**\*.zip<br/>Check if the package mentioned in the task is published as an artifact in the build or a previous stage and downloaded in the current job.
I do not understand what is wrong in my azure-pipelines.yml for artifact production...
Thanks
It seems that there is an issue where the published dlls are placed. Try the yaml below, that I have explicitly set the output directory for the published dlls and zipped the files after publish(that would probably be your next issue). I have also explicitly set in which folder to look for the published folder in order to publish the artifact.
trigger:
- master
pool:
vmImage: 'windows-latest'
steps:
- task: DotNetCoreCLI#2
inputs:
command: 'restore'
feedsToUse: 'select'
- task: DotNetCoreCLI#2
inputs:
command: 'build'
- task: DotNetCoreCLI#2
inputs:
command: 'publish'
publishWebProjects: true
modifyOutputPath: true
arguments: '--configuration $(BuildConfiguration) --output "$(build.artifactstagingdirectory)"'
zipAfterPublish: true
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)'
ArtifactName: 'artifact2'

Azure Devops Build Pipeline for solution with multiple project types

I have a solution that contains the following
Several Asp.net Projects (Microservices and Gateway)
.Net Core + Angular 8 (Front End)
When I hit the build button in Visual Studio every project is built. I have created a repo and uploaded the solution. Now I'm trying to figure out how to setup the pipeline to build each microservice and deploy them to individual Azure Web Apps.
I have the following Pipeline for the Angular Project. Should I define separate tasks like this? Is there a way to replicate the Visual Studio build here?
# ASP.NET
# Build and test ASP.NET projects.
# Add steps that publish symbols, save build artifacts, deploy, and more:
# https://learn.microsoft.com/azure/devops/pipelines/apps/aspnet/build-aspnet-4
trigger:
- master
pool:
vmImage: 'windows-latest'
variables:
solution: '**/*.sln'
buildPlatform: 'Any CPU'
buildConfiguration: 'Release'
steps:
- task: NuGetToolInstaller#1
- task: Npm#1
inputs:
command: install
workingDir: 'd:\a\1\s\Ok.Web\ClientApp'
- task: Npm#1
inputs:
command: custom
workingDir: 'd:\a\1\s\Ok.Web\ClientApp'
customCommand: run build
- task: CopyFiles#2
inputs:
targetFolder: '$(Build.ArtifactStagingDirectory)'
- task: PublishBuildArtifacts#1
You may apply one of two approaches here:
One pipeline for whole repo
One pipeline for project
In both cases you may use templates to avoid repeating yourself; so you will define common tasks for building a .NET project and then use them in pipelines. I recently made a blog post about this, but please take a look at the documentation too.
To achieve this you need to first define a YAML file with common steps. For instance:
parameters:
- name: buildConfiguration # name of the parameter; required
default: 'Release'
- name: projectFolder
default: ' '
steps:
- task: DotNetCoreCLI#2
displayName: Restore nuget packages
inputs:
command: restore
projects: '*.csproj'
workingDirectory: '${{ parameters.projectFolder}}'
- task: DotNetCoreCLI#2
displayName: Build
inputs:
command: build
projects: '${{ parameters.projectFolder}}'
arguments: '--configuration ${{ parameters.buildConfiguration }}'
# You just added coverlet.collector to use 'XPlat Code Coverage'
- task: DotNetCoreCLI#2
displayName: Test
inputs:
command: test
projects: '*Tests/*.csproj'
arguments: '--configuration ${{ parameters.buildConfiguration }} --collect:"XPlat Code Coverage" -- RunConfiguration.DisableAppDomain=true'
workingDirectory: '${{ parameters.projectFolder}}'
- task: DotNetCoreCLI#2
inputs:
command: custom
custom: tool
arguments: install --tool-path . dotnet-reportgenerator-globaltool
displayName: Install ReportGenerator tool
- script: ./reportgenerator -reports:$(Agent.TempDirectory)/**/coverage.cobertura.xml -targetdir:${{ parameters.projectFolder}}/coverlet/reports -reporttypes:"Cobertura"
displayName: Create reports
- task: PublishCodeCoverageResults#1
displayName: 'Publish code coverage'
inputs:
codeCoverageTool: Cobertura
summaryFileLocation: ${{ parameters.projectFolder}}/coverlet/reports/Cobertura.xml
And then invoke this file from you main build file:
variables:
buildConfiguration: 'Release'
projectFolder: 'path to your project'
steps:
- template: build-and-test.yaml
parameters:
buildConfiguration: $(buildConfiguration)
- script: echo Some steps to create artifacts!
displayName: 'Run a one-line script'
In approach number 1 you will build all projects even if you change just one project, so I would recommend you use approach number 2. For this you may define multiple pipelines (one per project) and limit triggers to changes in specific folder. Please take a look here.
Here you have an example of how you can limit triggers to specific folder for master branch:
trigger:
branches:
include:
- master
paths:
include:
- gated-checkin-with-template/*
exclude:
- gated-checkin-with-template/azure-pipelines-gc.yml

Deploy ASP.NET Core App Service with Azure DevOps path not found

I have a simple build pipeline that triggers on my git commit and is working great.
Here is the .yaml for that process:
trigger:
- master
pool:
vmImage: 'ubuntu-latest'
variables:
buildConfiguration: 'Release'
steps:
- script: dotnet build --configuration $(buildConfiguration)
- task: ArchiveFiles#2
inputs:
rootFolderOrFile: '$(Build.BinariesDirectory)'
includeRootFolder: true
archiveType: 'zip'
archiveFile: '$(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip'
replaceExistingArchive: true
displayName: 'dotnet build $(buildConfiguration)'
Then, I tried to make a Release build, but can't get the zip file across to my deploy step. The steps I follow are:
Publish build artifacts
Download build artifacts
Deploy web service
Here is my setup - Overall release pipeline:
Artifact Stage setup as follow:
Here is my 3 tasks in the Stage 1 (Deploy Stage):
Then the 3 tasks's properties:
And here is the error I'm getting, it is with regard to the artefact publish directory:
You should use dotnet publish to create your binaries. The step will also create a zip file. Then to publish the artefacts, use the PublishBuildArtifacts#1 Task. These steps should all be done within a build, not a release.
Here an example:
steps:
- script: dotnet build --configuration $(buildConfiguration)
displayName: 'dotnet build $(buildConfiguration)'
- task: DotNetCoreCLI#2
inputs:
command: 'publish'
publishWebProjects: false
projects: '**/*.csproj'
arguments: '-o /app'
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: '/app'
ArtifactName: 'drop'
publishLocation: 'Container'
Within the release, you don't need the Publish and Download Build Artefact step since the artefacts are already there (_ISOF). After you run the first Build, you can just select the zip file in the "Package or folder" dropdown.

Resources