I have enabled pipeline resource triggers between two pipelines. would like to replace alias value dynamically with triggering pipeline resource. below is the pipeline code
resources:
pipelines:
- pipeline: pipeline1
project: onecom
source: pipeline1-api
trigger:
branches:
- develop
- feat/*
- pipeline: pipeline2
project: onecom
source: pipeline2-api
trigger:
branches:
- develop
- feat
variables:
- name: apiname
value: $(resources.pipeline.<Alias>.pipelineName)
- name: dockertag
value: $(resources.pipeline.<Alias>.sourceCommit)
- name: runname
value: $(resources.pipeline.<Alias>.runName)
stages:
- stage: ScanImage
jobs:
- job: ScanImage
pool:
vmImage: 'ubuntu-16.04'
steps:
- script: echo $(apiname)
- script: echo $(runname)
I would like to replace Alias value in $(resources.pipeline..pipelineName) with value pipeline1 if build comes from source: pipeline1-api and with pipeline2 if build comes from source: pipeline2-api dynamically.
I would like to replace Alias value in $(resources.pipeline..pipelineName) with value pipeline1 if build comes from source: pipeline1-api and with pipeline2 if build comes from source: pipeline2-api dynamically.
Since the value of nested variables (like $(resources.pipeline.$(alias).pipelineName)) are not yet supported in the build/release pipelines. So we could not use it in the variable directly:
variables:
- name: apiname
value: $(resources.pipeline.$(alias).pipelineName)
To resolve this issue, we need add a inline powershell to set the variable resources.pipeline.<Alias>.pipelineName based on the value of the $(resources.triggeringAlias):
variables:
- name: alias
value: $(resources.triggeringAlias)
- task: InlinePowershell#1
inputs:
script: |
if ("$(alias)" -eq "PipelineA")
{
Write-Host ("##vso[task.setvariable variable=dockertag]$(resources.pipeline.PipelineA.sourceCommit) | cut -c -7")
}
elseif ("$(alias)" -eq "PipelineB")
{
Write-Host ("##vso[task.setvariable variable=dockertag]$(resources.pipeline.PipelineB.sourceCommit) | cut -c -7")
}
Update:
could you please help me same config in bash as we are using these
task in linux machines
- task: PowerShell#2
displayName: 'Inline Powershell'
inputs:
TargetType: inline
Script: |
if ("$(alias)" -eq "PipelineA")
{
Write-Host ("##vso[task.setvariable variable=dockertag]$(resources.pipeline.PipelineA.sourceCommit) | cut -c -7")
}
elseif ("$(alias)" -eq "PipelineB")
{
Write-Host ("##vso[task.setvariable variable=dockertag]$(resources.pipeline.PipelineB.sourceCommit) | cut -c -7")
}
pwsh: true
please try this
variables:
- name: alias
value: $(resources.triggeringAlias)
then you can try replace it as below
$(resources.pipeline.$(alias).pipelineName)
Related
I am trying to pass variables between stages. I was successfully able to pass values to immediate next stage, but after that the variables are blank. I tried all the ways to get this, but failing
stages:
- stage: InitialStage
jobs:
- job: StandAlone
displayName: Required Stand Alone Job
steps:
- bash: |
echo "##vso[task.setvariable variable=doThing;isOutput=true]Yes"
name: bash_test
- stage: SecondStage
jobs:
- job:
displayName: SecondStage
variables:
variable1: $[ stageDependencies.InitialStage.StandAlone.outputs['bash_test.doThing'] ]
steps:
- task: CmdLine#2
name: test
inputs:
script: |
echo "$(variable1)"
- stage: ThirdStage
jobs:
- job:
displayName: ThirdStage
variables:
variable1: $[ stageDependencies.InitialStage.StandAlone.outputs['bash_test.doThing'] ]
steps:
- task: CmdLine#2
name: test
inputs:
script: |
echo "$(variable1)"
The echo at stage SecondStage will give value, but ThirdStage it is blank, I am having a doubt either we can access a variable only in immediate stage or else I am doing something wrong, any help on this will be much appreciated.
Option 1:
In order to pass variable to the another stage you need to add isOutput=true
See the example as shown below;
- task: Bash#3
name: setprojectName
inputs:
targetType: 'inline'
script: |
echo "##vso[task.setvariable variable=projectNameGlobal;isOutput=true]$(echo $(Build.Repository.Name)| sed 's|.*/||')"
echo "##vso[task.setvariable variable=projectName]$(echo $(Build.Repository.Name)| sed 's|.*/||')"
projectName is only available the stage where variable populated but projectNameGlobal is available in next stage also.
Also see the documentation
Option 2:
## azure-pipelines.yml
stages:
- stage: one
jobs:
- job: A
steps:
- task: Bash#3
inputs:
filePath: 'script-a.sh'
name: setvar
- bash: |
echo "##vso[task.setvariable variable=skipsubsequent;isOutput=true]true"
name: skipstep
- stage: two
jobs:
- job: B
variables:
- name: StageSauce
value: $[ stageDependencies.one.A.outputs['setvar.sauce'] ]
- name: skipMe
value: $[ stageDependencies.one.A.outputs['skipstep.skipsubsequent'] ]
steps:
- task: Bash#3
inputs:
filePath: 'script-b.sh'
name: fileversion
env:
StageSauce: $(StageSauce) # predefined in variables section
skipMe: $(skipMe) # predefined in variables section
- task: Bash#3
inputs:
targetType: 'inline'
script: |
echo 'Hello inline version'
echo $(skipMe)
echo $(StageSauce)
In above option you can see how to use one variable where populated by any of the prior stages
is it possible to declare a variable and then pass it downstream?
I have an image below StageA -> StageB -> StageC
where I obtain a url for my storage account on StageA, and I want to use it for both Stage B and StageC
But if I use the [stagedependencies.StageA.JobA.outputs['var'], it only works on StageB and not on StageC
- stage: 'StageC'
dependsOn: 'StageB'
pool:
vmImage: 'windows-latest'
variables:
blobUri: $[stageDependencies.StageA.JobA.outputs['createOutput.blobUri']]
jobs:
- job: 'JobC'
steps:
- checkout: none
- download: none
- powershell: |
echo JobBUri: $(blobUri)
Maybe I missed it somewhere but does this mean you can only obtain the variable from the immediate stage you depend?
Example:
trigger:
- master
pool:
vmImage: ubuntu-latest
stages:
- stage: 'StageA'
jobs:
- job: 'JobA'
steps:
- task: Powershell#2
name: 'createOutput'
inputs:
targetType: 'inline'
script: |
Write-Output "##vso[task.setvariable variable=blobUri;isOutput=true]www.google.com"
- stage: 'StageB'
dependsOn: 'StageA'
pool:
vmImage: 'windows-latest'
variables:
blobUri: $[stageDependencies.StageA.JobA.outputs['createOutput.blobUri']]
jobs:
- job: 'JobB'
steps:
- powershell: |
echo JobBUri: $(blobUri)
- stage: 'StageC'
dependsOn: 'StageB'
pool:
vmImage: 'windows-latest'
variables:
blobUri: $[stageDependencies.StageA.JobA.outputs['createOutput.blobUri']]
jobs:
- job: 'JobC'
steps:
- powershell: |
echo JobBUri: $(blobUri)
Try this and probably variable set in stage A will be available at stage C by stageDependencies.StageA
- stage: 'StageC'
dependsOn:
- StageA
- StageB
Other workaround is to set again this variable as output variable in stage B and access it from C by stageDependencies.StageB(...)
From release notes
https://learn.microsoft.com/en-us/azure/devops/release-notes/2020/sprint-168-update#jobs-can-access-output-variables-from-previous-stages
By default, each stage in a pipeline depends on the one just before it in the YAML file. Therefore, each stage can use output variables from the prior stage. You can alter the dependency graph, which will also alter which output variables are available. For instance, if stage 3 needs a variable from stage 1, it will need to declare an explicit dependency on stage 1.
The following illustrates that the answer from Kontekst is correct (I was going to edit the post to add this, but don't know if that's appropriate here?):
The following confirms and illustrates that this is the correct answer:
stages:
- stage: A
jobs:
- job: A1
steps:
- pwsh: Write-Host "##vso[task.setvariable variable=varFromA;isOutput=true]A"
name: writevar
- stage: B
dependsOn: A
jobs:
- job: B1
variables:
varA: $[ stageDependencies.A.A1.outputs['writevar.varFromA'] ]
steps:
- pwsh: |
Write-Host "##vso[task.setvariable variable=varFromB;isOutput=true]B"
Write-Host "accesses $(VarA) from A"
name: writevar
- stage: C
dependsOn:
- A
- B
jobs:
- job: C1
variables:
varA: $[ stageDependencies.A.A1.outputs['writevar.varFromA'] ]
varB: $[ stageDependencies.B.B1.outputs['writevar.varFromB'] ]
steps:
- pwsh: |
Write-Host "##vso[task.setvariable variable=varFromC;isOutput=true]C"
Write-Host "accesses $(varB) from B"
Write-Host "accesses $(varA) from A"
name: writevar
I have a multistage YAML Azure pipeline. I tag my images based on the build ID. This works well until a job fails and I need to rerun it.
On the rerun I get a newly incremented build Id so it no longer references same docker image used in the original run.
$(Build.BuildId)
Is there anyway around this?
Re-running job or stage doesn't change Build.BuildId. I checked this using below pipeline. However if you want to run whole pipeline for the same build id you my try to use runtime paramaters and provide a tagName on your own (like below):
parameters:
- name: tagName
type: string
default: ' '
trigger: none
pr: none
pool:
vmImage: 'ubuntu-latest'
stages:
- stage: A
jobs:
- job: A
steps:
- pwsh: |
$tagName = '$(Build.BuildId)'
if('${{ parameters.tagName }}' -ne ' ') {
$tagName = '${{ parameters.tagName }}'
}
echo $tagName
- job: B
steps:
- bash: echo "B"
- stage: B
jobs:
- job: A
steps:
- pwsh: |
$tagName = '$(Build.BuildId)'
if('${{ parameters.tagName }}' -ne ' ') {
$tagName = '${{ parameters.tagName }}'
}
echo $tagName
- bash: exit 1
- job: B
steps:
- bash: echo "B"
You could expand stage and rerun the stage, this won't change the buildId:
I have a yml pipeline with 2 stages:
- stage:
- stage:
My second stage needs to refer to the $(Build.BuildNumber) of the previous stage. How can this be achieved? My understanding is output variables are scoped to the same stage and can't be used across-stages.
Trying to pull from stageDependencies:
stages:
- stage: BuildPublish
displayName: "Build & Publish"
jobs:
- job: BuildPublishJob
displayName: "Build & Publish"
steps:
- script: |
echo "Recording MSI version"
echo "##vso[task.setvariable variable=msiVersion;isOutput=true]$(Build.BuildNumber)"
name: MsiVersion
- script: echo $(MsiVersion.msiVersion)
name: echovar
- stage: DeployInstallerTest
displayName: "Deploy Installer Test"
jobs:
- job:
displayName: "Deploy Installer Test"
steps:
- task: AzurePowerShell#5
inputs:
azureSubscription: 'Spektrix Non-Prod'
ScriptType: 'InlineScript'
Inline: |
$msiVersion = stageDependencies.BuildPublish.BuildPublishJob.outputs['MsiVersion.msiVersion']
azurePowerShellVersion: 'LatestVersion'
Fails with:
##[error]The term 'stageDependencies.BuildPublish.BuildPublishJob.outputs[MsiVersion.msiVersion]' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
It changed recently:
stages:
- stage: A
jobs:
- job: JA
steps:
- script: |
echo "This is job Foo."
echo "##vso[task.setvariable variable=doThing;isOutput=true]Yes" #The variable doThing is set to true
name: DetermineResult
- script: echo $(DetermineResult.doThing)
name: echovar
- job: JA_2
dependsOn: JA
condition: eq(dependencies.JA.outputs['DetermineResult.doThing'], 'Yes')
steps:
- script: |
echo "This is job Bar."
#stage B runs if DetermineResult task set doThing variable n stage A
- stage: B
dependsOn: A
jobs:
- job: JB
condition: eq(stageDependencies.A.JA.outputs['DetermineResult.doThing'], 'Yes') #map doThing and check if true
variables:
varFromStageA: $[ stageDependencies.A.JA.outputs['DetermineResult.doThing'] ]
steps:
- bash: echo "Hello world stage B first job"
- script: echo $(varFromStageA)
However, please be aware that stageDependencies is not available in condition at stage level. Of course you can use stageDependencies not only in condition.
- task: PowerShell#2
displayName: Save Storage account Secrets to Build Variables
inputs:
azureSubscription:
targetType: 'inline'
script: '$outputs = ConvertFrom-Json $($env:STORAGE); foreach ($output in $outputs.PSObject.Properties) { echo $output.Name; echo $output.Value.value; Write-Host ("##vso[task.setvariable variable=$($output.Name);]$($output.Value.value)");}'
- phase: DEVRelease
dependsOn: Build
queue: Hosted Ubuntu 1604
steps:
- task: Kubernetes#1
displayName: Apply Kubernetes Deployment
inputs:
kubernetesServiceEndpoint:
arguments: "-f conf/deploy_local.yaml"
command: apply
azureSubscription:
azureContainerRegistry:
configMapName: myconfig
forceUpdateConfigMap: true
configMapArguments: --from-literal=myname=$($env:STORAGEACCOUNTNAME1)
Never reads the $env:STORAGEACCOUNTNAME variable
Since the PowerShell task to set the variables is in phase build. You need to add isOutput=true to the setvariable statement. Please check Set a multi-job output variable
"##vso[task.setvariable variable=$($output.name);isOutput=true]$($output.Value)"
I made a little bit changes to your yaml for testing. Please check it out. I have the env variable STORAGE = {'tags':[{'name':'A', 'Value':'1' }, { 'name':'B', 'Value':'2'}]}
phases:
- phase: build
queue: Hosted Ubuntu 1604
steps:
- powershell: |
$outputs = ConvertFrom-Json $($env:STORAGE)
foreach ($output in $outputs.tags) { echo $output.name; echo $output.Value; Write-Host ("##vso[task.setvariable variable=$($output.name);isOutput=true]$($output.Value)");}
name: myvariables
- powershell: |
echo "$(myvariables.A)"
echo "$(myvariables.A)"
- phase: DEVRelease
dependsOn: Build
queue: Hosted Ubuntu 1604
variables:
Da: $[ dependencies.build.outputs['myvariables.A'] ]
Db: $[ dependencies.build.outputs['myvariables.B'] ]
steps:
- powershell: |
echo $(Da)
echo $(Db)
In above script I output variable in phase build by adding isOutput=true to the statement, and give my powershell task a name name: myvariables.
And I refer to the output variable in the next phase DEVRelease by using statement $[ dependencies.{dependent phase name}.outputs['{task name}.{variable name}'] ] and assign it to the Variables.
Then i can successfully get the value in powershell task in phase DEVRelease.