I'm working with azure pipeline to checkout source code from azure repo and execute setup of inbuilt script to which is provided by webmethod SAG, Using build.yaml i can able to build my application but not able to publish the artifacts.
cat build.yaml
trigger:
- devops-build
pool:
name: CICD
steps:
# Create Target Directory to keep git repo for later use
- bash: |
mkdir -p /home/user/cicd_source/dev/packages/packages
displayName: 'create directory'
- bash: |
echo "webname=${{parameters.projectName}}" > $(Build.ArtifactStagingDirectory)/devpackagename.properties
echo "BuildNumber=$(Build.BuildNumber)" > $(Build.ArtifactStagingDirectory)/devBuildNumber.txt
Above script will create devpackagename.properties and devBuildNumber.txt following path inside my self hosted agent directory work location.
pwd
/home/user/agent/CICD/_work/1/a
ls -lrt
devpackagename.properties
devBuildNumber.txt
cat devpackagename.properties
webname=package
cat devBuildNumber.txt
BuildNumber=20221004.83
After ran the successful pipeline i don't see any artefacts published inside my pipeline
after your build steps add below task
- task: PublishPipelineArtifact#1
inputs:
targetPath: '$(Build.ArtifactStagingDirectory)'
artifact: 'drop'
publishLocation: 'pipeline'
you would see artifact get published on the pipeline
Related
I'm new to the Azure pipeline, right now working with WebMethod sag its a Middleware application to setup CI/CD using using Azure.
By default WebMethod Application provide the setup build script (ANT) using ant script we can able to build and deploy the application with application configuration properties files.
I have create azure pipeline YAML file to build and deploy my application. Please find the below sample build script.
trigger:
- none
pool:
name: default
parameters:
- name: projectName
displayName: appname?
type: string
steps:
- bash: |
mkdir -p /home/user/devops/azure-source/build/packages
displayName: 'create directory to copy source code'
- task: CopyFiles#2
displayName: 'Copy Files '
inputs:
SourceFolder: '${{parameters.projectName}}'
TargetFolder: '/home/user/devops/azure-source/build/packages'
- bash: |
mkdir -p /home/user/devops/azure-source/build/packages/$(Build.BuildNumber)
/home/viswa/softwareAg/ant/bin/build.sh
displayName: 'create build'
- bash: |
echo "name=packages" > $(Build.ArtifactDirectory)/packagename.properties
echo "BuildNumber=$(Build.BuildNumber)" > $(Build.ArtifactDirectory)/BuildNumber.txt
displayName: 'create package'
- task: PublishBuildArtifacts#1
displayName: 'Publish Artifact: drop'
After the successful build i can able to download the Artifacts which contains compiled source code and build number, i have attached sample output file.
BuildNumber.txt
BuildNumber=20220929.1
packagename.properties
name=myapp
Now the Question is I have to create another yaml lets name it as (rollback.yaml) with new pipeline and I want to rollback last build number (20220929.1) which contains last packagename (myapp) as well from my dev server.
can you please some one assist me how we can easily achieve it using azure artifacts with build number.
This is the reference doc I have followed to set up the Azure pipeline
https://medium.com/adessoturkey/owasp-zap-security-tests-in-azure-devops-fe891f5402a4
below i am sharing screenshort of the pipeline failed:
Could you please help here to resolve the issue I have exactly followed the medium article to implement the task....
Those who aware on this could you please share your taughts.
This is the pipeline script i am using.
trigger: none
stages:
stage: 'buildstage'
jobs:
job: buildjob
pool:
vmImage: ubuntu-latest
steps:
- checkout: self
- checkout: owasap-zap
bash: "docker run -d -p 80:80 nginx:1.14.2"
displayName: "App Container"
bash: |
chmod -R 777 ./
docker run --rm -v $(pwd):/zap/wrk/:rw -t owasp/zap2docker-stable zap-full-scan.py -t http://$(ip -f inet -o addr show docker0 | awk '{print $4}' | cut -d '/' -f 1):80 -x xml_report.xml
true
displayName: "Owasp Container Scan"
- displayName: "PowerShell Script"
powershell: |
$XslPath = "owasp-zap/xml_to_nunit.xslt"
$XmlInputPath = "xml_report.xml"
$XmlOutputPath = "converted_report.xml"
$XslTransform = New-Object System.Xml.Xsl.XslCompiledTransform
$XslTransform.Load($XslPath)
$XslTransform.Transform($XmlInputPath, $XmlOutputPath)
displayName: "PowerShell Script"
task: PublishTestResults#2
displayName: "Publish Test Results"
inputs:
testResultsFiles: converted_report.xml
testResultsFormat: NUnit
# task: PublishTestResults#2
stage: buildstage
According to the YAML file, you want to checkout multiple repositories in your pipeline, but it seems you haven't define a repository resource like mentioned in the document you shared.
resources:
repositories:
- repository: <repo_name>
type: git
name: <project_name>/<repo_name>
ref: refs/heads/master
And according to the screenshot you shared, you only checkout out one repo. Which cause the location of file xml_to_nunit.xslt is different from owasp-zap/xml_to_nunit.xslt. If you only checkout one repo, the location of xml_to_nunit.xslt should be current directory, thus, just define $XslPath in the PowerShell script as "xml_to_nunit.xslt".
Edit
If the repository that contain "xml_to_nunit.xslt" file is in the same organization as the repository run for your pipeline, you need to checkout the repository by using Inline syntax checkout like below or define repository resource.
- checkout: git://MyProject/MyRepo # Azure Repos Git repository in the same organization
You could also add one more command ls before the PowerShell script to list the files in current directory. Aim to figure out where is "xml_to_nunit.xslt".
I am using Azure DevOps to build a python wheel. I want to make it as generic as possible so that everyone in the team can use the same pipeline to build their own python wheels and deploying them in some databricks workspace. For that I need to know what the name of the file(s) in build pipeline output is to use it in my release pipeline.
Currently in this case the file is a python wheel saved in build pipeline output. I am using the following code in my pipeline yaml to publish it both at build pipeline output and an Azure artifact feed.
- task: PublishBuildArtifacts#1
inputs:
pathToPublish: '$(Build.ArtifactStagingDirectory)'
artifactName: dfordbx
- task: UniversalPackages#0
displayName: Publish
inputs:
command: publish
publishDirectory: $(Build.ArtifactStagingDirectory)/dist/
vstsFeedPublish: 'MyProject/py_artifacts_1732'
vstsFeedPackagePublish: $(packageBuildName)
After build pipeline run ends, there is one wheel file in dist folder. I need to get the name of this wheel. For my own code, I of course know the name. But when others run the pipeline for their code, this is not clear to me. I require to get this name in my release pipeline.
In other words, I am looking for a way in my yaml file to get "py_sample_package-0.6.5-py3-none-any.whl" name in the following structure:
By choosing published artifact:
Getting to the file:
The highlighted part is what I need to get in pipeline. Thank you.
Classic Release Pipelines
In a Classic Release pipeline, you reference a build artifact and use it as a trigger. Artifacts are downloaded to $(System.DefaultWorkingDirectory)\$(Build.DefinitionName).
To identify the .whl file in the artifact, add a powershell script into your release pipeline that identifies the file and creates a new variable using the ##vso logging syntax...
For example:
# find the first .whl file in the artifact folder
$whlFile = Get-ChildItem `
-Filter *.whl `
-Path "$(System.DefaultWorkingDirectory)\$(Build.DefinitionName)\dfordbx" |
ForEach-Object { $_.fullname } |
Select-Object -First 1
# create a variable with the full path to the file
Write-Host "##vso[task.setvariable variable=whlFile]$whlFile"
Now you can use the variable like any other defined pipeline variable $(whlFile)
Multi-Stage YAML Pipelines
If you're using a multi-stage YAML pipeline and need to use the artifact between stages, you can't assume that the file will be present on the machine because each job is potentially running on a different machine. You will need to download the artifact at the start of the job.
variables:
- artifactName: dfordbx
stages:
- stage: build
jobs:
- job: buildJob
steps:
- task: PublishPipelineArtifact#1
inputs:
targetPath: '$(Build.ArtifactStagingDirectory)'
artifactName: $(artifactName)
artifactType: 'pipeline'
# or use the 'publish' alias
- publish: '$(Build.ArtifactStagingDirectory)'
artifact: '$(artifactName)'
- stage: deploy
dependsOn: build
condition: success('build')
jobs:
- job: deployJob
steps:
- task: DownloadPipelineArtifact#1
inputs:
source: 'current'
artifact: $(artifactName)
path: '$(Pipeline.Workspace)/$(artifactName)'
# or use the 'download' alias
- download: current
artifact: $(artifactName)
- pwsh: |
$whlFile = Get-ChildItem -Path "$(Pipeline.Workspace)/$(artifactName)" -Filter *.whl |
ForEach-Object { $_.fullName } |
Select-Object -First 1
Write-Host "##vso[task.setvariable variable=whlFile]$whlFile"
when I run the build pipeline I am getting ##[error]File not found: 'git'. I have an agent running on a server. I installed Git on the server. The pipeline is using this agent and is tied to an Azure repo. I am using simple script as below. Please advice.
trigger:
- master
pool: 'build agent'
vmImage: 'ubuntu-latest'
steps:
- script: echo Hello, world!
displayName: 'Run a one-line script'
script: |
echo Add other tasks to build, test, and deploy your project.
echo See https://aka.ms/yaml
displayName: 'Run a multi-line script'
Here is all that you have to do if you want to create your Azure Pipeline:
Browse to Azure Pipelines and click on New Pipeline
Select Azure Repo when asked about the source of your codebase
Select your repository
Review your Pipeline YAML and click on Run
And voila, you have your first build running!
For customizing your build pipeline further, please check the various built-in build and release tasks.
Here is the YAML schema for your reference.
I have an azure devops multi-stage pipeline which requires 1 stage that copies files from another repository to $(build.artifactstagingdirectory)
For example my YAML looks like
trigger:
- master
resources:
- repo: self
variables:
tag: '$(Build.BuildId)'
stages:
- stage: Build
displayName: Build image
jobs:
- job: Build
...
- stage: Build
... define other resource/repository ...
- task: CopyFiles#2
inputs:
SourceFolder: 'k8s'
Contents: '**'
TargetFolder: '$(build.artifactstagingdirectory)'
This pipeline is connect to a repository, which is probably defined by repo: self. So the question is, can I change this repository for a specific stage?
You can run git command in powershell script. Add a step in your stage to execute git command like below example. In this way the other repo will be cloned to artifact directory.
- powershell: |
cd $(Build.artifactstagingdirectory)
git clone "https://<<Your PAT>>#dev.azure.com/_organization/_project/_git/_repo"
Note: use your personal access token (PAT) as authentication.