I was using azure-pipelines.yml for Azure Pipelines' Build. I included below scripts to make Artifact already which is used for Pipelines' Release.
# publish artifacts
- powershell: gci env:* | sort-object name | Format-Table -AutoSize | Out-File $env:BUILD_ARTIFACTSTAGINGDIRECTORY/environment-variables.txt
- task: PublishBuildArtifacts#1
inputs:
pathtoPublish: '$(Build.ArtifactStagingDirectory)'
artifactName: drop1
- task: DownloadBuildArtifacts#0
inputs:
buildType: 'current'
downloadType: 'single'
artifactName: 'drop2'
downloadPath: '$(System.ArtifactsDirectory)'
What should I set for Command under "Deploy to Kubernetes"?
How can I upload kubernetes .yaml file from GitHub to Artifacts? (what is script in azure-pipelines.yml?)
You can also add additional type of task to your existing Azure Build Pipeline, that will download another git repository content (your kubernetes.yaml files) in addition to default source, like this one:
- task: fakhrulhilal-maktum.GitDownloader.git-downloader.GitDownloader#0
It should put your GitHub repo content to $(Build.Repository.LocalPath), from where you can push it via another powershell-like task to $(Build.ArtifactStagingDirectory), like other community members mentioned.
Please note that fakhrulhilal-maktum.GitDownloader.git-downloader.GitDownloader is a custom task, so you will need first to install it to your DevOps Azure project.
Related
I am using Azure DevOps to build a python wheel. I want to make it as generic as possible so that everyone in the team can use the same pipeline to build their own python wheels and deploying them in some databricks workspace. For that I need to know what the name of the file(s) in build pipeline output is to use it in my release pipeline.
Currently in this case the file is a python wheel saved in build pipeline output. I am using the following code in my pipeline yaml to publish it both at build pipeline output and an Azure artifact feed.
- task: PublishBuildArtifacts#1
inputs:
pathToPublish: '$(Build.ArtifactStagingDirectory)'
artifactName: dfordbx
- task: UniversalPackages#0
displayName: Publish
inputs:
command: publish
publishDirectory: $(Build.ArtifactStagingDirectory)/dist/
vstsFeedPublish: 'MyProject/py_artifacts_1732'
vstsFeedPackagePublish: $(packageBuildName)
After build pipeline run ends, there is one wheel file in dist folder. I need to get the name of this wheel. For my own code, I of course know the name. But when others run the pipeline for their code, this is not clear to me. I require to get this name in my release pipeline.
In other words, I am looking for a way in my yaml file to get "py_sample_package-0.6.5-py3-none-any.whl" name in the following structure:
By choosing published artifact:
Getting to the file:
The highlighted part is what I need to get in pipeline. Thank you.
Classic Release Pipelines
In a Classic Release pipeline, you reference a build artifact and use it as a trigger. Artifacts are downloaded to $(System.DefaultWorkingDirectory)\$(Build.DefinitionName).
To identify the .whl file in the artifact, add a powershell script into your release pipeline that identifies the file and creates a new variable using the ##vso logging syntax...
For example:
# find the first .whl file in the artifact folder
$whlFile = Get-ChildItem `
-Filter *.whl `
-Path "$(System.DefaultWorkingDirectory)\$(Build.DefinitionName)\dfordbx" |
ForEach-Object { $_.fullname } |
Select-Object -First 1
# create a variable with the full path to the file
Write-Host "##vso[task.setvariable variable=whlFile]$whlFile"
Now you can use the variable like any other defined pipeline variable $(whlFile)
Multi-Stage YAML Pipelines
If you're using a multi-stage YAML pipeline and need to use the artifact between stages, you can't assume that the file will be present on the machine because each job is potentially running on a different machine. You will need to download the artifact at the start of the job.
variables:
- artifactName: dfordbx
stages:
- stage: build
jobs:
- job: buildJob
steps:
- task: PublishPipelineArtifact#1
inputs:
targetPath: '$(Build.ArtifactStagingDirectory)'
artifactName: $(artifactName)
artifactType: 'pipeline'
# or use the 'publish' alias
- publish: '$(Build.ArtifactStagingDirectory)'
artifact: '$(artifactName)'
- stage: deploy
dependsOn: build
condition: success('build')
jobs:
- job: deployJob
steps:
- task: DownloadPipelineArtifact#1
inputs:
source: 'current'
artifact: $(artifactName)
path: '$(Pipeline.Workspace)/$(artifactName)'
# or use the 'download' alias
- download: current
artifact: $(artifactName)
- pwsh: |
$whlFile = Get-ChildItem -Path "$(Pipeline.Workspace)/$(artifactName)" -Filter *.whl |
ForEach-Object { $_.fullName } |
Select-Object -First 1
Write-Host "##vso[task.setvariable variable=whlFile]$whlFile"
I need this scenario to be accomplish
Jenkins is the main ci/cd tool now we want to use Azure to be our build server but only for the build that is:
Jenkins checkout source
Run some build-in scripts to prepare for build
Send source which previously check out to the build server <-- this is where the AZURE part steps in
Copy the created artifacts back to Jenkins slave
Continue with CD on Jenkins slave
How do i combine the section 3 ?
How do i combine the section 3 ?
As far as I know, you can use Azure Devops self-hosted agent to connect azure devops and Jenkins.
You can refer to the following steps：
Create an Azure Devops Self-hosted agent on the Jenkins server.
Since the Checkout Step is on Jenkins, you could add Copy file task in Azure Devops to copy the Source to the Build Directory.
e.g. Source Repo Path -> $(Build.SourcesDirectory)
After the build step, you can copy the files back to the jenkins slave path.
e.g. Source: $(build.artifactstagingdirectory) -> Target: Slave Path
If your need to copy file to another remote machine, you could try to use Copy files over SSH or Windows Machine File Copy task.
Update:
After you configure the source, you could publish the repo as artifacts in Jenkins Server.
You could use the Jenkins download artifacts task in azure devops to download the artifacts.
Note: The artifacts will be downloaded to $(Build.ArtifactStagingDirectory).
Then you could run the task and publish the build artifacts on Hosted Macos Agent.
To Copy the created artifacts back to Jenkins:
You could add another Agent job and use the self-hosted agent(On linux AWS) to download the artifacts to jenkins server.
Agent Job 1 is running on Hosted MacOs agent. Agent Job 2 is ruuning on Self-hosted agent.
Update2:
Yaml Sample:
stages:
- stage: CopyFile
pool:
name: default
jobs:
- job: testjob
steps:
- task: CopyFiles#2
inputs:
SourceFolder: 'Local Path'
Contents: '**'
TargetFolder: '$(build.artifactstagingdirectory)'
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)'
ArtifactName: 'Source'
publishLocation: 'Container'
- stage: BuildProject
dependsOn: CopyFile
pool:
vmImage: ubuntu-16.04
jobs:
- job: buildjob
steps:
- task: DownloadBuildArtifacts#0
inputs:
buildType: 'current'
downloadType: 'single'
artifactName: 'Source'
downloadPath: '$(System.ArtifactsDirectory)'
- task: xxx(build task)
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)'
ArtifactName: 'Artifacts'
publishLocation: 'Container'
- stage: BackToJenkins
dependsOn: BuildProject
pool:
name: default
jobs:
- job: Sendjob
steps:
- task: DownloadBuildArtifacts#0
inputs:
buildType: 'current'
downloadType: 'single'
artifactName: 'Artifacts'
downloadPath: '$(System.ArtifactsDirectory)'
You need to modify the path(Publish Path, Download Path) to meet your needs.
I have a publish task in Azure Devops pipeline which should publish an excel file with extension .xlsx .
am using this below command but its not working. can someone help me with the wildcards?
this excel file is dynamic and date,month parameters keeps on changing.
format: SonarQube Issue Extract_2020-08-04.xlsx
task: PublishBuildArtifacts#1
inputs:
PathtoPublish: 'C:\Users\320066547\agent\_work\3\s\SonarFetchIssues\target\*.xlsx'
ArtifactName: 'IssuesOutput'
publishLocation: 'Container'
If using a YAML pipeline look at using the PublishPipelineArtifact task
- task: PublishPipelineArtifact#1
displayName: 'Publish Artifact: drop'
inputs:
artifact: drop
targetPath: $(Build.ArtifactStagingDirectory)
This will publish all the artifacts in the pipeline. From there can narrow down the exact path that is being published.
Alternatively too can do run a Powershell task before your publish to see exactly what your folder structure looks like on the agent:
-powershell: Get-ChildItem -Path 'Insert root path' -recurse
This is clearly stated in the task:
Path to publish: The folder or file path to publish. This can be a
fully-qualified path or a path relative to the root of the repository.
Wildcards are not supported. Variables are supported. Example: $(Build.ArtifactStagingDirectory)
I'm trying to deploy a .NET application using an Azure DevOps pipeline by making use of an ARM template and storing the template in an Azure git repository.
##################yml file for the pipeline build####################
- task: DotNetCoreCLI#2
inputs:
command: 'build'
projects: '**\*.csproj'
arguments: '-c Release'
- task: DotNetCoreCLI#2
inputs:
command: 'publish'
publishWebProjects: true
arguments: '-o $(Build.ArtifactStagingDirectory)/web'
- task: CopyFiles#2
inputs:
sourceFolder: 'appdeploy/'
TargetFolder: '$(Build.ArtifactStagingDirectory)/appdeploy'
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)'
ArtifactName: 'drop'
I am getting an error in the build step of the pipeline saying:
DotNetCoreCLI: If you're using Publish command with -o or --Output argument, you will see that the output folder is now being created at root directory rather than Project File's directory.
##[error]Project file(s) matching the specified pattern were not found.
According to the error message , this issue exists in the “build” step.
You need to define in the build task which .csproj files you want to build.
In your case, the definition path projects: '**\*.csproj' has some issues.
You could use the projects: '**/*.csproj’ to replace it.
Note: The cause of this issue is the backslash.
Hope this helps.
I have a published a build artifact published to $(Build.ArtifactStagingDirectory)/drop with artifactName "some_sidebar" and the artifact publish location is Azure Pipeline.
How can I retrieve that artifact now inside my release Pipeline if I have only a PowerShell Skript in release task?
here is the code specific part:
$path = ".\_some_sidebar\drop"
#$path = $(Build.Repository.LocalPath)
$SPFolderName = "Style Library/_some_sidebar";
# Upload template list
$status = "Uploading template list to Location: " + $SPFolderName
Write-Host $status
$te = Add-PnPFile -Path $path"\some_sidebar.js" -Folder $SPFolderName -Checkout
Set-PnPFileCheckedIn -Url $te.ServerRelativeUrl
I get the following error:
Uploading template list to Location: Style Library/_some_sidebar
2020-01-16T09:51:20.5062033Z Add-PnPFile : Local file was not found.
2020-01-16T09:51:20.5062546Z At D:\_work\_temp\6d682160-e8a7-4c56-ab30-7ff8c40f2958.ps1:51 char:7
2020-01-16T09:51:20.5062832Z + $te = Add-PnPFile -Path $path"\some_sidebar.js" -Folder $SPFolderName ...
I assume the build artifact path in azure pipeline is some path in the Virtual machine... but I don't know how to specify that path inside the shell script, or what that path is anyway...?
Azure DevOps: How to retrieve a build artifact from build Azure Pipeline from a PowerShell Script in Release Pipeline?
There are three questions in your post that cause this issue.
First, since you select the artifact publish location is Azure Pipeline, you could not set the targetPath. You could check the document Publish Build Artifacts task:
I assume what you said should be set the pathtoPublish to $(Build.ArtifactStagingDirectory)/drop with artifactName "some_sidebar"like:
But this pathtoPublish is used to set The folder or file path to publish, in other words, it is the artifact source location, not the target.
So, we do not need to use the \drop in the powershell scripts to get the artifact.
Second, MS provides a series of Release variables so that we can use them directly.
You could use the System.DefaultWorkingDirectory, System.ArtifactsDirectory or Agent.ReleaseDirectory:
So, we could use one of above three variables in the powershell scripts to get the artifact, but the variable not the full path to the file, it is the path for the artifact in the release pipeline, we need to do one more step.
Third, when you use release pipeline to get the artifact, which will set the artifact to the folder contain the Source alias:
As test, I create a sample with following powershell scripts:
$path = "$(System.DefaultWorkingDirectory)\<SourceAliasVlaue>\<AartifactName>"
#$path = $(Build.Repository.LocalPath)
$SPFolderName = "Style Library/_some_sidebar";
# Upload template list
$status = "Uploading template list to Location: " + $SPFolderName
Write-Host $status
Get-ChildItem -Path $path
I use the powershell scripts Get-ChildItem -Path $path to list the file in the artifact:
Now, I could get artifact file some_sidebar.js in the powershell task.
Note: You could try to use the wildcard to get the artifact, like:
$te = Add-PnPFile -Path "$(System.DefaultWorkingDirectory)\**\some_sidebar.js"
Hope this helps.
You should be able to use System.ArtifactsDirectory.
Here are my pipeline with example how I use the artifact from previous step. Same variable should be possible to use in powershell script.
(This example is from a yaml pipeline for build and release.)
stages:
- stage: build
displayName: 'Build and package solution'
jobs:
- job: buildsteps
displayName: 'Steps to build and package'
pool: 'PrivateVS2017'
steps:
- task: ArchiveFiles#2
inputs:
rootFolderOrFile: '$(Build.SourcesDirectory)/Web'
includeRootFolder: true
archiveType: 'zip'
archiveFile: '$(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip'
replaceExistingArchive: true
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)'
ArtifactName: 'it-service-wiki-build'
publishLocation: 'Container'
- stage: deploy_to_development
displayName: 'Deploy to development Environment'
dependsOn: build
jobs:
- deployment: deploy
displayName: 'Deploy the solution to Dev'
pool: 'PrivateVS2017'
environment: 'ITServiceWiki-Dev'
strategy:
runOnce:
deploy:
steps:
- task: DownloadBuildArtifacts#0
inputs:
buildType: 'current'
buildVersionToDownload: 'latest'
downloadType: 'single'
ArtifactName: 'it-service-wiki-build'
downloadPath: '$(System.ArtifactsDirectory)'
- task: ExtractFiles#1
inputs:
archiveFilePatterns: '../a/**/$(Build.BuildId).zip'
destinationFolder: '$(Build.DefaultWorkingDirectory)/$(Build.BuildId)'
cleanDestinationFolder: true
Note the ../a/**/ when searching for the zip after downloading the artifact. Not sure if /a/ is the same om all build agents can prob use system.artifactsDirectory here too.