Azure DevOps - Maven Pipeline publish artifacts - azure

I have a scala project, which packages (using maven) perfectly fine locally and also the build pipeline on azure devops works fine. Now I would like to somehow retrieve the produced .jar files in the target folder. Therefore I would like to publish the artifacts. But the .jar files can nowhere be found.
I have the following azure-pipelines.yaml, which copies/publishes the whole folder to the targetFolder, but there is no target folder, only the source code folder is copied. Now my question, how can one publish/access to published artifacts?
trigger:
- azure_devops_pipeline
pool:
vmImage: ubuntu-latest
stages:
- stage: Package
jobs:
- job: Package
steps:
- task: Maven#3
inputs:
mavenPomFile: 'pom.xml'
mavenOptions: '-Xmx3072m'
javaHomeOption: 'JDKVersion'
jdkVersionOption: '1.8'
jdkArchitectureOption: 'x64'
publishJUnitResults: false
mavenAuthenticateFeed: true
effectivePomSkip: true
goals: 'package'
options: '-Dmaven.test.skip=true -Pscala-2.12 -Pfat-jar'
#sonarQubeRunAnalysis: false
- stage: PublishArtifacts
jobs:
- job: PublishArtifacts
steps:
- task: CopyFiles#2
inputs:
#SourceFolder: '/home/vsts/work/1/s/target/'
Contents: '**'
TargetFolder: '$(build.artifactstagingdirectory)'
CleanTargetFolder: true
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)'
ArtifactName: 'drop'

Eeach job is a seprate machine. SO if you compile and produce files on one job they are not available on next job unless you will publish those files as artifact and download them on another artifact. So if you want to have this working please move them (steps from PublishArtifacts job) to your first job.

Related

CI-CD Azure devops for function app deployment

My Function app Name:
my_func_dev_app
Functions I have in my project
func_dev
func_prd
Push func_dev to my_func_dev_app
Push func_prd to my_func_prd_app
But It is pushing both func_dev and func_prd to my_func_de_app
My app folder structure
(folder)myapp_function
(folder)func_dev --__init__.py
-- function.json
(folder)func_prd --__init__.py
-- function.json
task.py
Multi stage Pipeline:
#pipelines1
trigger:
- '*'
stages:
- stage: 'Build'
displayName: 'Build the web application'
jobs:
- job: 'Build'
displayName: 'Build job'
pool:
name: 'onprem_agent'
steps:
- task: CopyFiles#2
displayName: 'Copy Files to: $(Build.ArtifactStagingDirectory)'
inputs:
SourceFolder: '$(Build.SourcesDirectory)'
Contents: |
**/*
!.git/**
OverWrite: true
CleanTargetFolder: true
TargetFolder: '$(Build.ArtifactStagingDirectory)'
- task: ArchiveFiles#2
displayName: "Archive files"
inputs:
rootFolderOrFile: '$(Build.ArtifactStagingDirectory)/myapp_function'
includeRootFolder: false
archiveFile: '$(Build.ArtifactStagingDirectory)/build$(Build.BuildId).zip'
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)'
ArtifactName: 'myapp_function'
- stage: 'DEV_AZURE'
displayName: 'Az DEV Deploy'
dependsOn: Build
condition: |
and
(
succeeded(),
eq(variables['Build.SourceBranchName'], variables['releaseBranchName'])
)
jobs:
- deployment: Deploy
environment: dev
variables:
- group: Release
strategy:
runOnce:
deploy:
steps:
- download: current
artifact: myapp_function
displayName: Downloading artifacts
- task: AzureFunctionApp#1
inputs:
azureSubscription: CI-CD
appType: functionAppLinux
appName: my_func_dev_app
package: '$(Pipeline.Workspace)/myapp_function/*.zip'
deployToSlotOrASE: true
resourceGroupName: my-rg
slotName: PRODUCTION
displayName: 'Deploying dev'
How to fix? Push func_dev to my_func_dev_app and ignore func_prd
How to fix? Push func_dev to my_func_dev_app and ignore func_prd
To solve this issue, you can package your two projects separately into zip.
In your ArchiveFiles task, you need to specify a specific project.
For example:
For func_dev project: $(Build.ArtifactStagingDirectory)/myapp_function/func_dev
- task: ArchiveFiles#2
displayName: "Archive files"
inputs:
rootFolderOrFile: '$(Build.ArtifactStagingDirectory)/myapp_function/func_dev'
includeRootFolder: false
archiveFile: '$(Build.ArtifactStagingDirectory)/build$(Build.BuildId)-func_dev.zip'
Then you can only package the project corresponding to the function app name.
If you want to deploy two projects to the corresponding function app in one pipeline at the same time, you could use the same method to separately package the two projects as zip.

Excluding Certain Files in Azure CI Pipeline (YAML)

I have a CI pipeline (YAML) that builds a repo that will deploy into an existing Azure Function. The CI pipeline is doing it is job. However, after it is done, and I go to Function App -> App files -> I can see the azure-pipeline.yml is included in there (or i think it was included in the build process). I have tried using paths and exclude but they dont work. My question is, how do I exclude only that azure-pipeline.yml so that after the pipeline is done building, the azure-pipeline.yml is not in App files in Function App. Below is my YAML
trigger:
branches:
include:
- master
paths:
exclude:
- README.md
- azure-pipelines.yml
variables:
# Azure Resource Manager connection created during pipeline creation
azureSubscription: 'DevOps-Test'
# Function app name
functionAppName: 'test'
# Agent VM image name
vmImageName: 'vs2017-win2016'
# Working Directory
workingDirectory: '$(System.DefaultWorkingDirectory)/'
stages:
- stage: Build
displayName: Build stage
jobs:
- job: Build
displayName: Build
pool:
vmImage: $(vmImageName)
steps:
- powershell: |
if (Test-Path "extensions.csproj") {
dotnet build extensions.csproj --output ./$(workingDirectory)/bin
}
displayName: 'Build extensions'
- task: ArchiveFiles#2
displayName: 'Archive files'
inputs:
rootFolderOrFile: $(workingDirectory)
includeRootFolder: false
archiveType: zip
archiveFile: $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
replaceExistingArchive: true
- publish: $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
artifact: drop
- task: DownloadBuildArtifacts#0
inputs:
buildType: 'current'
downloadType: 'single'
artifactName: 'drop'
downloadPath: '$(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip'
- task: AzureFunctionApp#1
displayName: 'Azure functions app deploy'
inputs:
azureSubscription: '$(azureSubscription)'
appType: functionApp
appName: $(functionAppName)
package: '$(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip'
The following syntax means file README.md or azure-pipelines.yml won't trigger the build. It doesn't mean file README.md or azure-pipelines.yml are excluded in the working directory.
trigger:
branches:
include:
- master
paths:
exclude:
- README.md
- azure-pipelines.yml
I've noticed you tried archiving folder $(workingDirectory), and workingDirectory defined in variable which was actually $(System.DefaultWorkingDirectory)/. System.DefaultWorkingDirectory is the local path on the agent where your source code files are downloaded.
Obviously, file README.md and azure-pipelines.yml are in the source code, so they are archived too. You could add a CopyFiles task before ArchiveFiles task to copy files you need from a source folder to a target folder using match patterns, then archive the target folder. For example:
- task: CopyFiles#2
displayName: 'Copy Files to: $(Build.ArtifactStagingDirectory) '
inputs:
SourceFolder: '$(workingDirectory)'
Contents: |
**/*
!*.md
!*.yml
TargetFolder: '$(Build.ArtifactStagingDirectory) '
- task: ArchiveFiles#2
displayName: 'Archive files '
inputs:
rootFolderOrFile: '$(Build.ArtifactStagingDirectory) '
Take a look here
- powershell: |
if (Test-Path "extensions.csproj") {
dotnet build extensions.csproj --output ./$(workingDirectory)/bin
}
displayName: 'Build extensions'
- task: ArchiveFiles#2
displayName: 'Archive files'
inputs:
rootFolderOrFile: $(workingDirectory)
includeRootFolder: false
archiveType: zip
archiveFile: $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
replaceExistingArchive: true
You produce your ouptut here ./$(workingDirectory)/bin but you zipped rootFolderOrFile: $(workingDirectory). Please change it to rootFolderOrFile: $(workingDirectory)/bin.
EDIT
Please add this before calling archive
- script: |
rm README.md
rm azure-pipelines.yml
workingDirectory: $(workingDirectory)
So you will remove them from folder which you later archive.

How to resolve "unexpected value 'stages' azure pipelines" in Azure devops pipeline yml

I am new to Azure devops. as part of poc, i am trying to build a java based docker image.
so i have following pipeline yaml file
# Maven
# Build your Java project and run tests with Apache Maven.
# Add steps that analyze code, save build artifacts, deploy, and more:
# https://learn.microsoft.com/azure/devops/pipelines/languages/java
trigger:
- master
pool:
vmImage: 'ubuntu-latest'
steps:
- task: Maven#3
inputs:
mavenPomFile: 'pom.xml'
mavenOptions: '-Xmx3072m'
javaHomeOption: 'JDKVersion'
jdkVersionOption: '1.8'
jdkArchitectureOption: 'x64'
publishJUnitResults: true
testResultsFiles: '**/surefire-reports/TEST-*.xml'
goals: 'package'
resources:
- repo: self
variables:
tag: '$(Build.BuildId)'
stages:
- stage: Build
displayName: Build image
jobs:
- job: Build
displayName: Build
pool:
vmImage: 'ubuntu-latest'
steps:
- task: Docker#2
displayName: Build an image
inputs:
command: build
dockerfile: '$(Build.SourcesDirectory)/Dockerfile'
tags: |
$(tag)
Expected
What i expected, this pipeline need to create a java application (jar file) and then it should create a docker image using this jar
Actual:
i am getting below error
unexpected value 'stages' azure pipelines
I didnt understand the issue...
Appreciated if anybody can help on this..?
Thanks
Please move this into job before your first task - task: Docker#2
steps:
- task: Maven#3
inputs:
mavenPomFile: 'pom.xml'
mavenOptions: '-Xmx3072m'
javaHomeOption: 'JDKVersion'
jdkVersionOption: '1.8'
jdkArchitectureOption: 'x64'
publishJUnitResults: true
testResultsFiles: '**/surefire-reports/TEST-*.xml'
goals: 'package'
This is simply syntax issue. If you use steps you cant use stages on the root level. Here you may check syntax.
If you want to have jar creation as separate stage/job you have to define explicitly this as another stage or job. However in that way you need to publish and then download your jar as pipeline artifact.
If you stay with one single job you may just use:
trigger:
- master
pool:
vmImage: 'ubuntu-latest'
variables:
tag: '$(Build.BuildId)'
steps:
- task: Maven#3
inputs:
mavenPomFile: 'pom.xml'
mavenOptions: '-Xmx3072m'
javaHomeOption: 'JDKVersion'
jdkVersionOption: '1.8'
jdkArchitectureOption: 'x64'
publishJUnitResults: true
testResultsFiles: '**/surefire-reports/TEST-*.xml'
goals: 'package'
- task: Docker#2
displayName: Build an image
inputs:
command: build
dockerfile: '$(Build.SourcesDirectory)/Dockerfile'
tags: |
$(tag)

Does Azure DevOps Pipelines cache some data accross runs

I'm new to Azure DevOps and pipelines, and I ran into an issue running the same pipeline multiple times in a short period.
In brief, I created a pipeline to simply build a .Net project with MSBuild and generate an artifact. The pipeline trigger on change in master branch.
The first time, it worked, I can download the artifact and execute the program without any issue. Now if I do a change in the master branch 5 minutes later adding an option to my program, the pipeline runs successfully, however when running program stored in the generated artifact, my new option is not there.
I'm probably doing something stupid there, but I don't understand why I have this behaviour.
Is there any kind of caching and how can I have fresh build everytime ?
== EDIT ==
Here is my YAML definition as requested
Basically, steps are:
Checkout solution with all submodule
Nuget restore packages for all required projects
MSBuild task
Archive the output
Publish artifact.
trigger:
- master
pool:
demands: azureps
vmImage: 'windows-latest'
steps:
- checkout: "git://GSS-CMDB-Tools/GSSAM_Code"
submodules: true
persistCredentials: true
- task: NuGetCommand#2
inputs:
command: 'custom'
arguments: 'restore ADDMSync/packages.config -SolutionDirectory .'
- task: NuGetCommand#2
inputs:
command: 'custom'
arguments: 'restore GSSAM/packages.config -SolutionDirectory .'
- task: NuGetCommand#2
inputs:
command: 'custom'
arguments: 'restore GSSAM.ADDMRest/packages.config -SolutionDirectory .'
- task: NuGetCommand#2
inputs:
command: 'custom'
arguments: 'restore GSSAM.SNOWRest/packages.config -SolutionDirectory .'
- task: MSBuild#1
inputs:
solution: 'ADDMSync/ADDMSync.csproj'
msbuildArchitecture: 'x64'
configuration: 'Release'
msbuildArguments: '/p:PostBuildEvent='
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
# Write your PowerShell commands here.
mv ADDMSync/bin/Release ADDMSync/Bin/ADDMSync
rm ADDMSync/bin/ADDMSync/*.pdb
- task: ArchiveFiles#2
inputs:
rootFolderOrFile: 'ADDMSync/bin/ADDMSync'
includeRootFolder: true
archiveType: 'zip'
archiveFile: '$(Build.ArtifactStagingDirectory)/ADDMSync.zip'
replaceExistingArchive: true
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)/ADDMSync.zip'
ArtifactName: 'ADDMSync'
publishLocation: 'Container'
Thanks a lot
RĂ©mi
OK I think I understand what happens.
What I did was to commit and push all submodules required by the build. However I did not commit the modification of the solution itself. By doing so it makes it working.
I don't understand why for now, I guess it's link to the way the checkout task works.

Azure DevOps Artifacts

I am new to Azure DevOps (Hosted Agent) and am trying to use the Azure Pipelines to build my Java web app using Ant
below is the pipeline file
trigger:
- azure-pipelines
pool:
vmImage: 'ubuntu-latest'
steps:
- task: Ant#1
inputs:
workingDirectory: ''
buildFile: 'ant/build.xml'
javaHomeOption: 'JDKVersion'
jdkVersionOption: '1.8'
jdkArchitectureOption: 'x64'
publishJUnitResults: true
testResultsFiles: '**/TEST-*.xml'
Project got build successfully but the path of the WAR file is shown as
Building war: /home/vsts/work/1/s/war/Project.war
I am not able find /home/vsts/work/1/s path in Azure DevOps, I tried to search under Artifact but not found, How can I access the /home/vsts/work/1/s so that I can get my WAR file?
You need to use Publish Artifact task in order to be able to obtain the results of your build.
- task: PublishBuildArtifacts#1
inputs:
pathtoPublish: '$(Build.ArtifactStagingDirectory)' # or your build results directory
artifactName: 'drop'
#publishLocation: 'Container' # Options: container, filePath
#targetPath: # Required when publishLocation == FilePath
#parallel: false # Optional
#parallelCount: # Optional
#fileCopyOptions: #Optional

Resources