Recently just built an Azure Pipeline where in one stage there are different zip files in the artifact staging directory. What I'm trying to achieve is publish to the drop folder all the zip files from the staging folder with PublishPipelineArtifact task.
I have 2 archived zip files in artifact staging directory:
$(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
$(Build.ArtifactStagingDirectory)/cli_scripts_$(Build.BuildId).zip
In my azure-pipelines.yml file please find the publish task:
- task: PublishPipelineArtifact#0
displayName: 'Publish pipeline artifacts'
inputs:
targetPath: $(Build.ArtifactStagingDirectory)/**
This gives the following error:
[error] Path does not exist: d:\a\1\a**
I have already tried with the following as well but none of them working:
$(Build.ArtifactStagingDirectory)/**
$(Build.ArtifactStagingDirectory)/**/*.zip
$(Build.ArtifactStagingDirectory)/*.zip
Question:
What is the pattern for targetPath to move all the zip files from that folder?
Any help is appreciated!
What finally resolved the issue is including a pattern with archiveFilePatterns in the task and not combining with the targetPath as I originally tried.
The solution which worked well is the following:
- task: PublishPipelineArtifact#0
displayName: 'Publish pipeline artifacts'
inputs:
targetPath: $(Build.ArtifactStagingDirectory)/
archiveFilePatterns: '**/*.zip'
The official documentation does not really states this but it was giving the idea using the pattern attribute: Publish and download artifacts
I hope that helps someone in the future.
You can use the .artifactignore file to filter what the PublishPipelineArtifact task can see. Make sure the file is in the folder that you're publishing, as mentioned here:
Related
I want to publish a java jar file to azure devops artifacts section in azure devops. Once it is there, I want to be able to setup my gradle so it can download jars from that location. I think I understand how to do the second step, but how to do the first step is not clear to me. I can use an azure pipelines task to build the jar and publish the artifact. However the published artifact does not end up in the location I want. It ends up in an artifacts section of the pipeline run, it does not end up in the artifacts section of ADO. Here is the code:
steps:
- checkout: self
path: 'my-repo/'
- task: Gradle#2
displayName: Test and build jar
inputs:
gradleWrapperFile: 'gradlew'
tasks: 'build'
publishJUnitResults: false
javaHomeOption: 'JDKVersion'
- task: PublishBuildArtifacts#1
inputs:
pathToPublish: '$(System.DefaultWorkingDirectory)/build/libs/commons.jar'
You can see in this image. The green arrow points to where the artifact ends up after successful execution of the above code. However when I click on the artifacts section indicated by the blue arrow it is not there. How do I get it to publish over there so it can be consumed by my build process on my local machine? Thanks in advance.
PublishBuildArtifacts#1 publishes 'Build artifacts' which are later available to be downloaded by DownloadBuildArtifacts#0 in later pipeline stages or in another pipelines. (And it will be only accessible where your green arrow points)
'Build artifact' is not the same thing as 'Artifacts Feed'
'Artifacts Feed' serves as source of packages which can be downloaded for example by NuGet if you configure in nuget.exe as a source your 'Artifacts Feed'
For example you can add NuGet package there using NuGet push(with firstly configured Source)
https://learn.microsoft.com/en-us/azure/devops/artifacts/get-started-nuget?view=azure-devops&tabs=windows#publish-a-nuget-package-by-using-the-command-line
To publish Universal Packages there is already task UniversalPackages#0
https://learn.microsoft.com/en-us/azure/devops/pipelines/artifacts/universal-packages?toc=%2Fazure%2Fdevops%2Fartifacts%2Ftoc.json&bc=%2Fazure%2Fdevops%2Fartifacts%2Fbreadcrumb%2Ftoc.json&view=azure-devops&tabs=yaml
I have two pipelines: one creates binary files, the other one uses this binary file.
For this, I need to share the files created by the first pipeline to the other one.
I tried using the Publish Artifact task, but I get an error message when trying to download the artifacts in the second pipeline.
This is the pipeline configuration for the first pipeline:
steps:
#this script creates the binaries
- script: do something
- publish: $(System.DefaultWorkingDirectory)/path/to/folder
artifact: artifact1
This is the pipeline configuration for the second pipeline:
steps:
- download: current
artifact: artifact1
This second pipeline fails with the following error: ##[error]Artifact artifact1 was not found for build 123.
I think the reason for this error might be that published artifacts are only available in jobs of the same pipeline run. Is this correct?
If so then how can I share these files?
You need to put specific instead of current if you are downloading from a artifact published by another pipeline:
steps:
- download: specific
artifact: artifact1
Check this out:
- task: DownloadPipelineArtifact#2
inputs:
source: 'specific'
project: 'FabrikamFiber'
pipeline: 12
runVersion: 'latest'
https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/utility/download-pipeline-artifact?view=azure-devops#download-artifacts-from-a-specific-projectpipeline
I have a publish task in Azure Devops pipeline which should publish an excel file with extension .xlsx .
am using this below command but its not working. can someone help me with the wildcards?
this excel file is dynamic and date,month parameters keeps on changing.
format: SonarQube Issue Extract_2020-08-04.xlsx
task: PublishBuildArtifacts#1
inputs:
PathtoPublish: 'C:\Users\320066547\agent\_work\3\s\SonarFetchIssues\target\*.xlsx'
ArtifactName: 'IssuesOutput'
publishLocation: 'Container'
If using a YAML pipeline look at using the PublishPipelineArtifact task
- task: PublishPipelineArtifact#1
displayName: 'Publish Artifact: drop'
inputs:
artifact: drop
targetPath: $(Build.ArtifactStagingDirectory)
This will publish all the artifacts in the pipeline. From there can narrow down the exact path that is being published.
Alternatively too can do run a Powershell task before your publish to see exactly what your folder structure looks like on the agent:
-powershell: Get-ChildItem -Path 'Insert root path' -recurse
This is clearly stated in the task:
Path to publish: The folder or file path to publish. This can be a
fully-qualified path or a path relative to the root of the repository.
Wildcards are not supported. Variables are supported. Example: $(Build.ArtifactStagingDirectory)
I have been having issues with sending build artifacts to my feed and can't figure out where my issue is at.
I forked over this repository from an Azure document since I am new to this and learning to create a CI/CD pipeline (https://github.com/Azure-Samples/python-docs-hello-world).
With the twine or universal package publishing setup guides there are steps for connecting to the feed such as creating a .piyrc file in your home directory but is that done locally or somewhere within the pipeline YAML?
Universal Publishing YAML
steps:
- task: UniversalPackages#0
displayName: 'Universal publish'
inputs:
command: publish
vstsFeed: 'cd75ead1-7beb-42f9-9477-e958501bb986'
publishDirectory: '$(Pipeline.Workspace)'
vstsFeedPublish: 'cd75ead1-7beb-42f9-9477-e958501bb986'
vstsFeedPackagePublish: drop
Twine Method
twine upload -r {Feed} --config-file $(PYPIRC_PATH) $(Pipeline.Workspace)
With Universal Publishing I receive an error about the path provided as being invalid.
With Twine I get an error about InvalidDistribution: Cannot find file (or expand pattern)
The $(Pipeline.Workspace) that I have written above was created as a path in my build pipeline to copy all files over from an Archive step. I see the artifact being made in the build pipeline and then downloaded in the first step of the release pipeline so I'm not sure what is going on or if it's something as simple as using the incorrect path.
With Twine I get an error about InvalidDistribution: Cannot find file (or expand pattern)
You need to specify the specific artifacts path instead of using the $(Pipeline.Workspace).
The $(pipeline.workspcae) is equal to the $(Agent.BuildDirectory). You could refer to this doc.
From the Github link, it seems that you want to publish a python package to feed.
You could refer to the following steps to create CI\CD.
In CI , you could Build sdist and publish the artifact to pipeline.
Here is the sample:
steps:
- task: UsePythonVersion#0
displayName: 'Use Python 3.6'
inputs:
versionSpec: 3.6
- script: 'python setup.py sdist'
displayName: 'Build sdist'
- task: PublishBuildArtifacts#1
displayName: 'Publish Artifact: dist'
inputs:
PathtoPublish: dist
ArtifactName: dist
In CD , you could set the build artifacts as resource and use twine to upload the python package to feed.
Here is an example:
twine upload -r AzureTest23 --config-file $(PYPIRC_PATH) D:\a\r1\a\{Source alias}\dist\*
The twine authenticate task could give the $(PYPIRC_PATH) variable.
If you want to determine your correct path, you can find it in the release log.
Note: If there are spaces or special characters in the path, they need to be escaped in cmd, otherwise they will not be recognized.
The name relates with the source alias, you could change it in artifacts source.
By the way, if you use the Universal Publish task, you also need to give the correct path.
I would like to copy all the .nupkg files recursively, but only from directories which their name start with "Zafiro.".
I've created this task in the azure-pipelines.yml
- task: CopyFiles#2
inputs:
SourceFolder: '$(Agent.BuildDirectory)'
Contents: '**\Zafiro.*\*.nupkg'
TargetFolder: '$(build.artifactstagingdirectory)'
flattenFolders: true
But it copies 0 files.
It seems the wildcard ('**\Zafiro.**.nupkg') isn't working as expected.
How do I get what I want?
Azure Pipelines Wildcard advice
The pattern Zafiro.* should be work. I test it with a sample in my side, and it works fine.
As test, I upload some nuget packages in different folder include the test folder Zafiro.Test in the repo directly instead of create the nuget packages:
Now, I use the same YAML file to test it and in order to see the result more intuitively, I replace the TargetFolder to my local folder:
pool:
name: MyPrivateAgent
steps:
- task: CopyFiles#2
inputs:
SourceFolder: '$(Agent.BuildDirectory)'
Contents: '**\Zafiro*\*.nupkg'
TargetFolder: 'D:\PublishFolder'
flattenFolders: true
Then the result:
So, the pattern Zafiro.* should be work.
To resolve this issue, first, we need to make sure we have the nuget package in the Zafiro.* in the Agent.BuildDirectory folder. We could change the Contents '**\Zafiro*\*.nupkg' to '**\*.nupkg' to check if we have nuget package in the Agent.BuildDirectory folder first.
Second, just like Shayki Abramczyk said, we could try to remove . in the Zafiro.*.
If above still not resolve your question, please share the debug log for this copy task.