I have two pipelines: one creates binary files, the other one uses this binary file.
For this, I need to share the files created by the first pipeline to the other one.
I tried using the Publish Artifact task, but I get an error message when trying to download the artifacts in the second pipeline.
This is the pipeline configuration for the first pipeline:
steps:
#this script creates the binaries
- script: do something
- publish: $(System.DefaultWorkingDirectory)/path/to/folder
artifact: artifact1
This is the pipeline configuration for the second pipeline:
steps:
- download: current
artifact: artifact1
This second pipeline fails with the following error: ##[error]Artifact artifact1 was not found for build 123.
I think the reason for this error might be that published artifacts are only available in jobs of the same pipeline run. Is this correct?
If so then how can I share these files?
You need to put specific instead of current if you are downloading from a artifact published by another pipeline:
steps:
- download: specific
artifact: artifact1
Check this out:
- task: DownloadPipelineArtifact#2
inputs:
source: 'specific'
project: 'FabrikamFiber'
pipeline: 12
runVersion: 'latest'
https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/utility/download-pipeline-artifact?view=azure-devops#download-artifacts-from-a-specific-projectpipeline
Related
My azure repo has multiple projects in it (it is not a single solution with multiple projects), which are placed in subfolders. How to publish artifacts separately for each of the project, which are placed in different folders.
By using paths include statements you can specify a build file and a trigger for each of the subdirectories.
An example below
trigger:
branches:
include:
- refs/heads/Development
paths:
include:
- slapi
- slapi.Common
batch: True
name: $(date:yyyyMMdd)$(rev:.r)
This allows each subdirectory to trigger and build to a separate output directory as needed.
In Azure Devops you then import this build YAML file to create your build pipeline and then when you create your release pipeline you select this build pipeline as the source for the drop files
Obviously you need to complete the above example file with the actual build configuration but since the question was about using subdirectories I included the relevant parts.
For your working directories use the variables section like this to specify the project at hand
variables:
vmImageName: "windows-2022"
workingDirectory: "projectName" <- change this to your real project name
then refer to the variables in your build task like this
- task: DotNetCoreCLI#2
displayName: Build
inputs:
command: "build"
projects: |
$(workingDirectory)/*.csproj
arguments: --output $(System.DefaultWorkingDirectory)/publish_output --configuration Release
I can't get my yaml pipeline to trigger based off the build completion of another pipeline.
This is the code from the yaml pipeline I am trying to trigger to run, where pipelineX is the name of the pipeline build I want the run to trigger off of:
resources:
pipelines:
- pipeline: trigger-pipeline
source: pipelineX
trigger: true
steps:
- task: Bash#3
inputs:
targetType: 'inline'
script: |
echo 'pipeline runs here'
Both pipelines are part of the same project, so that should not be an issue.
I have similar setup with my project. I am triggering 'PipelineNeedsTrigger' build based on successful build of 'PipelineTriggerFrom' pipeline. Both pipelines are under the same project. I have exported my YAML and added snippet here which might help you. I have added below trigger into my 'PipelineNeedsTrigger' which will trigger once the 'PipelineTriggerFrom' has successful build on 'master' branch. This might help you link.
resources:
pipelines:
- pipeline: PipelineTriggerFrom
source: PipelineTriggerFrom
trigger:
branches:
include:
- refs/heads/master
I am trying to improve the CI/CD process of an old funky project whose code is not open to refactoring at the moment. I just cant get this to work following the Azure documentation or even if it is possible.
I have been able to improve the current state with an azure pipeline file that runs unit tests before merging into releases/dev branch. But i want to further.
Tasks every PR into releases/dev will:
script: npm run test:unit
script: npm run build:dev
copy/publish the contents of the .div/ folder to a azure blob store config for static site
Any PR or merge into releases/staging will:
script: npm run build:staging
copy/publish the contents of the .div/ folder to a azure blob store config for static site
Any PR or merge into master will:
script: npm run test:unit
script: npm run build:production
copy/publish the contents of the .div/ folder to a azure blob store config for static site
I have 3 questions
Is this possible within a single yaml file?
How do i run different task for different branches, I've defined jobs/stages but cant get them to be conditional?
Is there some magic anyone can direct me to that lets me copy the content of a directory to a blob store? Or must it be zipped->copied->un zipped?
Thanks in advance from a new sleep deprived dad
Is this possible within a single yaml file? How do i run different
task for different branches, I've defined jobs/stages but cant get
them to be conditional?
Of course. You could add these stages in a single yaml file. Then you need to define the condiftion field for each stage or each job.
Here is an example for stages:
trigger:
- '*'
pool:
vmImage: 'ubuntu-latest'
stages:
- stage: Test1
condition: OR(eq(variables['Build.SourceBranch'], 'refs/heads/master') ,eq(variables['System.PullRequest.TargetBranch'], 'refs/heads/master'))
jobs:
- job: BuildJob
steps:
- script: echo Build Stage1!
- stage: Test2
condition: OR(eq(variables['Build.SourceBranch'], 'refs/heads/dev') ,eq(variables['System.PullRequest.TargetBranch'], 'refs/heads/dev'))
jobs:
- job: BuildJob
steps:
- script: echo Build Stage2!
- stage: Test3
condition: OR(eq(variables['Build.SourceBranch'], 'refs/heads/staging') ,eq(variables['System.PullRequest.TargetBranch'], 'refs/heads/staging'))
jobs:
- job: BuildJob
steps:
- script: echo Build Stage3!
You could set required branches as trigger. Then you could use the Build.SourceBranch and System.PullRequest.TargetBranch to determine whether to run the current stage.
Build.SourceBranch -> for merge branch.
System.PullRequest.TargetBranch -> for Pull Request.
Here are the docs about conditions and variables.
Is there some magic anyone can direct me to that lets me copy the content of a directory to a blob store? Or must it be zipped->copied->un zipped?
Since you need to publish file to Azure Blob, you could directly try to use the Azure File Copy task.
Here is an example:
- task: AzureFileCopy#4
displayName: 'AzureBlob File Copy'
inputs:
SourcePath: xxx
azureSubscription: xxx
Destination: AzureBlob
storage: xxx
ContainerName: '$web'
Hope this helps.
Recently just built an Azure Pipeline where in one stage there are different zip files in the artifact staging directory. What I'm trying to achieve is publish to the drop folder all the zip files from the staging folder with PublishPipelineArtifact task.
I have 2 archived zip files in artifact staging directory:
$(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
$(Build.ArtifactStagingDirectory)/cli_scripts_$(Build.BuildId).zip
In my azure-pipelines.yml file please find the publish task:
- task: PublishPipelineArtifact#0
displayName: 'Publish pipeline artifacts'
inputs:
targetPath: $(Build.ArtifactStagingDirectory)/**
This gives the following error:
[error] Path does not exist: d:\a\1\a**
I have already tried with the following as well but none of them working:
$(Build.ArtifactStagingDirectory)/**
$(Build.ArtifactStagingDirectory)/**/*.zip
$(Build.ArtifactStagingDirectory)/*.zip
Question:
What is the pattern for targetPath to move all the zip files from that folder?
Any help is appreciated!
What finally resolved the issue is including a pattern with archiveFilePatterns in the task and not combining with the targetPath as I originally tried.
The solution which worked well is the following:
- task: PublishPipelineArtifact#0
displayName: 'Publish pipeline artifacts'
inputs:
targetPath: $(Build.ArtifactStagingDirectory)/
archiveFilePatterns: '**/*.zip'
The official documentation does not really states this but it was giving the idea using the pattern attribute: Publish and download artifacts
I hope that helps someone in the future.
You can use the .artifactignore file to filter what the PublishPipelineArtifact task can see. Make sure the file is in the folder that you're publishing, as mentioned here:
I'm setting up several pipelines in Azure DevOps. To make my teams life easier, I'm using job templates.
These job templates are in a a proper repository, just for them.
For every pipeline I define the repository to get the templates from.
Some tasks in these templates run powershell code, and I want this code to be in a script file, to be reusable and stored in the same repo as the template.
When the pipelines runs, the template is embeded, it tries to locate the powershell script inside project repo actually being built/deployed.
How can i achieve this?
The workaround is to have inline code which I really don't want to have.
Any constructive answer will be very appreciated.
Thanks
After some digging I couldn't find any way to specify a script file as source to powershell task in a template.
Inside pipeline definition:
resources:
repositories:
- repository: templates
type: git
name: deploy-templates
variables:
artifactName: 'Trade Data ETL - $(Build.SourceBranchName)'
stages:
- stage: Build
displayName: Build
variables:
- group: DEV-Credential-Group
- group: COMMON-Settings-Group
jobs:
- template: ssis/pipelines/stage-build.yml#templates # Template reference
parameters:
artifactName: '$(artifactName)'
Inside template file:
- task: PowerShell#2
inputs:
filePath: ssis/pipelines/scripts/build-ssis-project.ps1
arguments: '-ProjectToBuild "tradedata-ldz-ssis/tradedata-ldz-ssis.dtproj'
pwsh: true
Update 2021
According to learn.microsoft.com, you can now also check out multiple repositories without custom scripting.
If you check out more than one repository, a separate folder containing the repository is created below $(Build.SourcesDirectory).
You can define multiple repositories like this:
resources:
repositories:
- repository: devops
type: git
name: DevOps
ref: main
- repository: infrastructure
type: git
name: Infrastructure
ref: main
And in the steps simply check them out as follows:
steps:
- checkout: self
- checkout: devops
- checkout: infrastructure
# List all available repositories
- script: ls
Original Answer
Currently the resources command only supports yml files in other repositories. However, you could simply checkout the repository in a task and then run the desired powershell script.
steps:
- task: PowerShell#2
inputs:
targetType: inline
script: |
git clone -b <your-desired-branch> https://azuredevops:$($env:token)#dev.azure.com/<your-organization>/<your-project>/_git/<your-repository> <target-folder-name>
./<target-folder-name>/foo.ps1
env:
token: $(System.AccessToken)
This script would checkout an arbitrary branch and execute a script foo.ps1 in the root of the target repository.
Call - checkout: templates inside the template file. This might only work when you insert a template but it successfully sees the repository resource and pulls it down.
You can copy the script files from source directory. Currently, you have not mentioned the root folder -
ssis/pipelines/scripts/build-ssis-project.ps1
Assuming, you are building on a repo where the powershell script resides -
Try -
- task: PowerShell#1
inputs:
scriptName: '$(ScriptsDir)/ssis/pipelines/scripts/build-ssis-project.ps1'
Pass the value of ScriptsDir where it could be the build source directory or build working directory