Return an object variable from a script - Azure YAML pipelines - azure

Consider the following simplified pipeline:
### template.yml ###
parameters:
- name: "tables"
type: object
default: {}
steps:
- ${{ each table in parameters.tables }}:
- task: BackupTask#0
displayName: "Backup ${{ table }}"
### pipeline.yml ###
- template: template.yml
parameters:
tables:
- "table1"
- "table2"
- "table3"
- "table4"
- "table5"
What I would like is that the list of tables are generated with a bash script instead of having to write them by hand. So every time a new table is created it gets automatically backed up by the pipeline.

As a workaround, we can create another pipeline. In this pipeline, we add two powershell tasks. In the first task, we set a variable with tables as the value.
- task: PowerShell#2
inputs:
targetType: 'inline'
script: 'Write-Host "##vso[task.setvariable variable=list]table1,table2,table3"'
In the second task, we use rest api to trigger the pipeline.yml pipeline. In the request body, we use the variable set in the first task as the value of the template parameter.
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
$token = "PAT"
$url="https://dev.azure.com/{org}/{pro}/_apis/pipelines/{pipelineId}/runs?api-version=5.1-preview"
$token = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes(":$($token)"))
$JSON = #'
{
"templateParameters": {
"tab":"[$(list)]"
},
}
'#
$response = Invoke-RestMethod -Uri $url -Headers #{Authorization = "Basic $token"} -Method Post -Body $JSON -ContentType application/json
Below is my test sample:
### pipeline.yml ###
parameters:
- name: tab
type: object
default: {}
pool:
vmImage: 'ubuntu-latest'
steps:
- template: template1.yml
parameters:
tables: ${{ parameters.tab }}
template.yml:
### template1.yml ###
parameters:
- name: "tables"
type: object
default: {}
steps:
- ${{ each table in parameters.tables }}:
- task: PowerShell#2
inputs:
targetType: 'inline'
script: echo "${{ table }}"
Then we run the newly created pipeline to trigger the pipeline.yml pipeline, get the result:

Related

Is there a way to handle a single parameter as a build number OR source branch in an azure pipeline?

I'm trying to allow for a single variable that can specify the build id or branch to deploy from. This means I need to type coerce, and it's failing if I try to call a ge (greater than or equal) or lt (less than) on a string. It looks like I might have to use bash commands to do the type conversion for me, so I was wondering if anyone had this handy.
Here's the steps I'm using:
- ${{ each deployment in parameters.deployments }}:
- deployment: Deploy_${{deployment.serviceName}}_${{ parameters.region }}
pool:
vmImage: ${{ parameters.vmImage }}
displayName: 'Deploy ${{ deployment.serviceName }} ${{ parameters.region }}'
${{ if not(eq(parameters.kubernetesServiceEndpoint, '')) }}:
environment: ${{ parameters.kubernetesServiceEndpoint }}
${{ elseif not(and(eq(parameters.azureResourceGroup, ''), eq(parameters.kubernetesCluster, ''))) }}:
environment: ${{ parameters.environment }}
strategy:
runOnce:
deploy:
steps:
# Disable the automatic downloading of artifacts, because we will specifying exactly what we need
- download: none
- task: DownloadPipelineArtifact#2
condition: ${{ ge(0, deployment.branchBuildId) }}
inputs:
source: 'specific'
project: ${{ deployment.project }}
pipeline: ${{ deployment.pipeline }}
runVersion: 'latestFromBranch'
runBranch: ${{ deployment.branchBuildId }}
patterns: 'DeploymentData/*'
displayName: 'Download Latest Artifacts'
- task: DownloadPipelineArtifact#2
condition: ${{ lt(0, deployment.branchBuildId) }}
inputs:
source: 'specific'
project: ${{ deployment.project }}
pipeline: ${{ deployment.pipeline }}
runVersion: 'specific'
runId: ${{ deployment.branchBuildId }}
#runBranch: ${{ deployment.branch }}
patterns: 'DeploymentData/*'
displayName: 'Download Specific Artifacts (${{ deployment.buildId }})'
I want these to be mutually exclusive (run the first if pulling the latest from a specific branch, and run the second if downloading from a specific build id). By using a single parameter I can avoid another control parameter to decide which version should be used.
Two things:
1, If you use script to handle, then 'compile time usage' is unable to use.
You need to use runtime usage:
variables['xxx']
2, Variables only support passing string, you can compare them and then use logging command to output the result:
trigger:
- none
pool:
vmImage: ubuntu-latest
parameters:
- name: str1
default: 110
- name: str2
default: 130
steps:
- task: PowerShell#2
name: outputresult
inputs:
targetType: 'inline'
script: |
$str1 = "${{parameters.str1}}"
$str2 = "${{parameters.str2}}"
Write-Host $str1
Write-Host $str2
# powershell compare the strings greater than or less than
if ($str1 -gt $str2) {
$result = "greater than"
Write-Host $result
Write-Host "##vso[task.setvariable variable=result;isoutput=true]$result"
} elseif ($str1 -lt $str2) {
$result = "less than"
Write-Host $result
Write-Host "##vso[task.setvariable variable=result;isoutput=true]$result"
} else {
$result = "equal"
Write-Host $result
Write-Host "##vso[task.setvariable variable=result;isoutput=true]$result"
}
#Below is the handling logic
- task: PowerShell#2
condition: eq('greater than',variables['outputresult.result'])
inputs:
targetType: 'inline'
script: |
# Write your PowerShell commands here.
Write-Host "greater than"
- task: PowerShell#2
condition: eq('less than',variables['outputresult.result'])
inputs:
targetType: 'inline'
script: |
# Write your PowerShell commands here.
Write-Host "less than"
Result:

AzureDevOps Multistage build and release pipeline variable sharing

What am I doing wrong? I've got a multistage build and release pipeline that works in the following way
Build > release to UAT > release to UAT 2 (our prod) but I'm just playing around to get it working
trigger:
- master
pool:
vmImage: 'windows-latest'
variables:
solution: '**/*.sln'
buildPlatform: 'Any CPU'
buildConfiguration: 'Release'
stages:
- stage: Build
displayName: 'Build'
variables:
solution: '**/*.sln'
buildPlatform: 'Any CPU'
buildConfiguration: 'Release'
jobs:
- job: Build
steps:
- task: NuGetToolInstaller#0
displayName: 'Use NuGet 5.x'
inputs:
versionSpec: 5.x
- task: NuGetCommand#2
displayName: 'NuGet restore'
inputs:
restoreSolution: '**/*.sln'
- task: VSBuild#1
displayName: 'Build solution'
inputs:
solution: '$(solution)'
msbuildArgs: '/p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:PackageLocation="$(build.ArtifactStagingDirectory)\\"'
platform: '$(buildPlatform)'
configuration: '$(buildConfiguration)'
- task: PublishSymbols#2
displayName: 'Publish symbols path'
inputs:
SearchPattern: '**\bin\**\*.pdb'
PublishSymbols: false
continueOnError: true
- task: PublishBuildArtifacts#1
displayName: 'Publish Artifact'
inputs:
PathtoPublish: '$(build.ArtifactStagingDirectory)'
ArtifactName: 'drop'
- stage: UAT
displayName: 'UAT'
condition: and(succeeded(), eq(variables['Build.SourceBranchName'], 'master'))
dependsOn: Build
jobs:
- deployment: DeployApp
pool:
vmImage: 'windows-latest'
displayName: 'Release artefacts to Test-Evn (Zip)'
environment: 'test-env'
strategy:
runOnce:
deploy:
steps:
- task: DownloadPipelineArtifact#2
displayName: 'Download Artifacts (drop)'
inputs:
buildType: 'current'
artifactName: 'drop'
targetPath: '$(System.ArtifactsDirectory)'
- task: PowerShell#2
condition: and(succeeded(), not(canceled()))
name: RetainOnSuccess
displayName: Retain on Success
inputs:
failOnStderr: true
targetType: 'inline'
script: |
$contentType = "application/json";
$headers = #{ Authorization = 'Bearer $(System.AccessToken)' };
$rawRequest = #{ daysValid = 365; definitionId = $(System.DefinitionId); ownerId = 'User:$(Build.RequestedForId)'; protectPipeline = $false; runId = $(Build.BuildId) };
$request = ConvertTo-Json #($rawRequest);
$uri = "$(System.CollectionUri)$(System.TeamProject)/_apis/build/retention/leases?api-version=6.0-preview.1";
$newLease = Invoke-RestMethod -uri $uri -method POST -Headers $headers -ContentType $contentType -Body $request;
$newLeaseId = $newLease.Value[0].LeaseId
echo "##vso[task.setvariable variable=newLeaseId;isOutput=true]$newLeaseId";
- script: echo $(RetainOnSuccess.newLeaseId)
name: echovar
- stage: UAT2
displayName: 'UAT2'
condition: and(succeeded(), eq(variables['Build.SourceBranchName'], 'master'))
dependsOn: UAT
jobs:
- deployment: DeployApp
pool:
vmImage: 'windows-latest'
displayName: 'Release artefacts to TestEnv2 (Zip)'
environment: 'test-env-2'
variables:
- name: NewLeaseId
value: $[ dependencies.UAT.outputs['DeployApp.RetainOnSuccess.newLeaseId']]
strategy:
runOnce:
deploy:
steps:
- task: DownloadPipelineArtifact#2
displayName: 'Download Artifacts (drop)'
inputs:
buildType: 'current'
artifactName: 'drop'
targetPath: '$(System.ArtifactsDirectory)'
- script: echo "https://dev.azure.com/testing-ci/$(System.TeamProject)/_apis/build/retention/leases/$(NewLeaseId)?api-version=6.1-preview.2"
- task: PowerShell#2
condition: and(succeeded(), not(canceled()))
name: RetainOnSuccess
displayName: Retain on Success
inputs:
failOnStderr: true
targetType: 'inline'
script: |
$contentType = "application/json";
$headers = #{ Authorization = 'Bearer $(System.AccessToken)' };
$rawRequest = #{ daysValid = 365 * 100; leaseId = $newLeaseId; ownerId = 'User:$(Build.RequestedForId)' };
$request = ConvertTo-Json #($rawRequest);
$uri = "https://dev.azure.com/testing-ci/$(System.TeamProject)/_apis/build/retention/leases/$(NewLeaseId)?api-version=6.1-preview.2";
Invoke-RestMethod -uri $uri -method PATCH -Headers $headers -ContentType $contentType -Body $request;
I'm trying to update the retention lock as per documentation
If a release gets put to UAt then apply a lock for a year
If a release gets to prod then update that retention indef
the error I'm getting is:
[error]Invoke-RestMethod : {"count":1,"value":{"Message":"The requested resource does not support http method 'PATCH'."}}
That's because its missing the value of the variable and therefore the url that its trying to patch to is
https://dev.azure.com/testing-ci/$(System.TeamProject)/_apis/build/retention/leases/?api-version=6.1-preview.2"
It's missing the value of $(NewLeaseId)
$[ dependencies.UAT.DeployApp.outputs['RetainOnSuccess.newLeaseId'] ]

Is there a way to loop over all outputs from stageDependencies in Azure DevOps?

I want to loop over all the outputs of dependent stages and check if any of them are set to a specific value which would indicate there was a problem. Is this possible, and if so, how?
I'm having trouble finding any relevant documentation on stageDependencies outside of using them in job conditions. I know the code below doesn't work, but I've tried a few variations on this and I'm not even seeing the stageDependencies object.
- task: PowerShell#2
displayName: Check for Failures
inputs:
targetType: inline
script: |
foreach ($dependency in $[stageDependencies]) {
foreach ($job in $dependency) {
foreach ($output in $job.outputs) {
if ("$output" -eq "Started") {
write-host "##vso[task.logissue type=error]Task failed in a previous step"
}
}
}
}
The pipeline can output variables through the logging command, but the variables can only be strings:
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops&tabs=yaml%2Cbatch#set-variables-in-scripts
All variables set by this method are treated as strings.
It is not difficult to parse a string and put it into an array, just use the function(split()) that comes with the string type to split and restore.
Here is an example:
trigger:
- none
# 1
stages:
- stage: s1
displayName: setvars
jobs:
- job: testJob
steps:
- task: PowerShell#2
name: setvar
inputs:
targetType: 'inline'
script: |
# logic here. For example you get the vars and put it into this format:
$testvars = "testvar1,testvar2,testvar3"
Write-Host "##vso[task.setvariable variable=outputvars;isOutput=true]$testvars"
# 2
- stage: s2
displayName: getvars
dependsOn: s1
variables:
vars: $[ stageDependencies.s1.testJob.outputs['setvar.outputvars'] ]
jobs:
- job:
steps:
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
$varsArr = "$(vars)".Split(',')
foreach ($var in $varsArr)
{
Write-Host "$var`r`n"
if($var -eq 'testvar1')
{
Write-Host 'The value is correct.'
}
}
Result:

Set variable on azure task to pass another task of Azure pipeline

I am able to set a variable on powershell or bash script and pass the variable to another task by using ##vso[task.setvariable variable=abc;]xyz. But couldn't find any documentation to set variable on azure tasks like azure webapp deploy task or SqlAzureDacpacDeployment task. I want to catch the error by passing the variable value. Is there any effective way to catch the azure task error log for the next task?
- task: SqlAzureDacpacDeployment#1
displayName: 'Insertion SQL Task'
inputs:
azureSubscription: 'Org (xxxxx-xxxx-xxxx-xxxx-xxxx)'
ServerName: 'tcp:abc.database.windows.net'
DatabaseName: test_db
SqlUsername: '$(user)'
SqlPassword: $(pass)
deployType: SqlTask
SqlFile: 'SQL/test.sql'
enabled: true
- task: AzureWebApp#1
displayName: 'Azure Web App Deploy: $(AppName)'
inputs:
azureSubscription: 'Org (xxxxx-xxxx-xxxx-xxxx-xxxx)'
appType: webApp
ResourceGroupName: 'Test'
appName: '$(AppName)'
package: '$(Build.ArtifactStagingDirectory)\app/*.zip'
deploymentMethod: zipDeploy
enabled: true
You will need to set a name for the task.
For Azure SQL Database Deployment task, you can use the SqlDeploymentOutputFile output variable name.
- task: SqlAzureDacpacDeployment#1
displayName: 'Insertion SQL Task'
name: sqlInsert
inputs:
...
enabled: true
- script: echo "$(sqlInsert.SqlDeploymentOutputFile)"
The Azure Web App task does not provide the same mechanism. You could always call the Pipelines - Get Logs API to get what you need:
pool:
vmImage: ubuntu-latest
steps:
- powershell: Write-Host "Hello World"
name: 'HelloWorld'
- powershell: |
Write-Host "This is a script that could use $env:SYSTEM_ACCESSTOKEN"
# Construct the REST URL to obtain Build ID
$uri = "$(System.CollectionUri)$(System.TeamProject)/_apis/pipelines/$(System.DefinitionId)/runs/$(Build.BuildId)/logs?api-version=6.0-preview.1"
Write-Host "$uri"
$base64AuthInfo = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f "system", $env:SYSTEM_ACCESSTOKEN)))
$logs = Invoke-RestMethod -Uri $uri -Method Get -ContentType "application/json" -Headers #{Authorization=("Basic {0}" -f $base64AuthInfo)}
env:
SYSTEM_ACCESSTOKEN: $(System.AccessToken)

Azure Devops : Set a variable for future stages and Job not working

I am trying to generate a random string of length 32 characters in Build Stage, then I want to pass the same to the next job/deployment And Also to the next Stage.
I referred to https://learn.microsoft.com/en-us/azure/devops/pipelines/process/set-variables-scripts?view=azure-devops&tabs=powershell to pass a variable from one job to another job and stage. But it is not working, Here is my pipeline code YAML file.
trigger: none
pool:
name: agentpool-myproj
stages:
- stage: 'BuildStage'
variables:
- group: myproj-vargrp-common-dp-poc
displayName: 'BuildStage'
jobs:
- deployment: BuildStageSecretProducerjobs # deployment is equal to job
environment: dev
displayName: 'BuildStageSecretProducerjobs'
strategy:
runOnce:
deploy:
steps:
- checkout: none
- download: none
- task: AzurePowerShell#5
name: mastertask
inputs:
azureSubscription: 'con-myproj-dev'
ScriptType: 'InlineScript'
Inline: |
$Random32Key = -join (((48..57)+(65..90)+(97..122)) |Get-Random -Count 32 |%{[char]$_})
Write-Host "##vso[task.setvariable variable=supersecret;isoutput=true;issecret=false]$Random32Key"
FailOnStandardError: true
azurePowerShellVersion: LatestVersion
- deployment: BuildStageSecretConsumerjobs
dependsOn: BuildStageSecretProducerjobs
variables:
thissupersecret: $[ dependencies.BuildStageSecretProducerjobs.outputs['masterjob.supersecret'] ]
environment: dev
displayName: 'BuildStageSecretConsumer'
strategy:
runOnce:
deploy:
steps:
- checkout: none
- download: none
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
# Write your PowerShell commands here.
Write-Host $(thissupersecret)
- stage: 'DeployStage'
dependsOn: 'BuildStage'
condition: succeeded('BuildStage')
displayName: 'DeployStage'
jobs:
- deployment: DeployStageSecretconsumerjobs # deployment is equal to job
variables:
thissupersecret: $[stageDependencies.BuildStage.BuildStageSecretProducerjobs.outputs['mastertask.supersecret']]
environment: dev
displayName: 'DeployStageSecretconsumerjobs'
strategy:
runOnce:
deploy:
steps:
- checkout: none
- download: none
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
# Write your PowerShell commands here.
Write-Host $(thissupersecret)
$releaseurl = ('{0}{1}/_apis/release/releases/{2}?api-version=5.0' -f $($env:SYSTEM_TEAMFOUNDATIONSERVERURI), $($env:SYSTEM_TEAMPROJECTID), $($env:RELEASE_RELEASEID) )
$Release = Invoke-RestMethod -Uri $releaseurl -Headers #{Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"}
$variableName = '<HERE_VARIABLE_NAME>'
$release.variables.($variableName).value = "<YOUR VALUE HERE IN STRING OR REFERNCE TO A VARIABLE E.G. $(OTHER_VARIABLE)>"
$json = #($release) | ConvertTo-Json -Depth 99
Invoke-RestMethod -Uri $releaseurl -Method Put -Body $json -ContentType "application/json" -Headers #{Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN" }
$Release = Invoke-RestMethod -Uri $releaseurl -Headers #{Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"}
Above is 100% working for Releases in ADO. Try to dig for URLs referencing Builds and you should be fine. Of course is you opt to Powershell.

Resources