I have a problem with the Azure Pipelines, whose behaviour during deployment I cannot explain at the moment.
I have two pipeline yaml's. azure-pipeline-api.yml and azure-pipeline-client.yml for the same AzureRepo which contains a Api and a Client.
There are two branches, one is main and the other is dev. I would like the pipelines to use different variables in the Helm deployments depending on the branch.
However, I have the problem that no one of the HelmDeployment gets variables. Although both pipelines are written identically. We can be sure that the variables passed to the stage are all filled and correct. (I wrote the job ReadVariable to check this).
API:
trigger:
- main
- dev
pool:
name: test-pool
stages:
- stage: Api
jobs:
- job: DefineVariables
displayName: Define Variables
steps:
- task: Bash#3
displayName: Set env variables
name: EnvVariables
inputs:
targetType: "inline"
script: |
PACKAGE_VERSION_API=$(xmlstarlet select --template -value-of /Project/PropertyGroup/Version --nl "TestProject/TestProjectApi.csproj")
if [ $(Build.SourceBranch) == "refs/heads/main" ]
then
IS_PROD=true
API_VERSION=$PACKAGE_VERSION_API
SERVICE_CONNECTION=aws-prod
SERVICE_NAMESPACE=test-prod
DEPLOYMENT_FILE_API=devops/deployments/api/prod-values.yaml
else
IS_PROD=false
API_VERSION=$PACKAGE_VERSION_API-dev
SERVICE_CONNECTION=aws-dev
SERVICE_NAMESPACE=test-dev
DEPLOYMENT_FILE_API=devops/deployments/api/dev-values.yaml
fi
echo "##vso[task.setvariable variable=IS_PROD;isOutput=true;]$IS_PROD"
echo "##vso[task.setvariable variable=API_VERSION;isOutput=true;]$API_VERSION"
echo "##vso[task.setvariable variable=SERVICE_CONNECTION;isOutput=true;]$SERVICE_CONNECTION"
echo "##vso[task.setvariable variable=SERVICE_NAMESPACE;isOutput=true;]$SERVICE_NAMESPACE"
echo "##vso[task.setvariable variable=DEPLOYMENT_FILE_API;isOutput=true;]$DEPLOYMENT_FILE_API"
echo Build.Reason: $(Build.Reason)
echo Build.SourceBranch: $(Build.SourceBranch)
echo IS_PROD: $IS_PROD
echo API_VERSION: $API_VERSION
echo SERVICE_CONNECTION: $SERVICE_CONNECTION
echo SERVICE_NAMESPACE: $SERVICE_NAMESPACE
echo DEPLOYMENT_FILE_API: $DEPLOYMENT_FILE_API
- job: ReadVariable
dependsOn: ["DefineVariables"]
displayName: Read env variables
variables:
API_VERSION: $[ dependencies.DefineVariables.outputs['EnvVariables.API_VERSION'] ]
IS_PROD: $[ dependencies.DefineVariables.outputs['EnvVariables.IS_PROD'] ]
SERVICE_CONNECTION: $[ dependencies.DefineVariables.outputs['EnvVariables.SERVICE_CONNECTION'] ]
SERVICE_NAMESPACE: $[ dependencies.DefineVariables.outputs['EnvVariables.SERVICE_NAMESPACE'] ]
DEPLOYMENT_FILE_API: $[ dependencies.DefineVariables.outputs['EnvVariables.DEPLOYMENT_FILE_API'] ]
steps:
- task: Bash#3
inputs:
targetType: "inline"
script: |
echo Build.Reason: $(Build.Reason)
echo Build.SourceBranch: $(Build.SourceBranch)
echo IS_PROD: $(IS_PROD)
echo API_VERSION: $(API_VERSION)
echo SERVICE_CONNECTION: $(SERVICE_CONNECTION)
echo SERVICE_NAMESPACE: $(SERVICE_NAMESPACE)
echo DEPLOYMENT_FILE_API: $(DEPLOYMENT_FILE_API)
- job: BuildApi
dependsOn: ["DefineVariables"]
variables:
API_VERSION: $[ dependencies.DefineVariables.outputs['EnvVariables.API_VERSION'] ]
steps:
- task: Docker#2
displayName: "Api - Build and Push Docker Image"
inputs:
containerRegistry: "dockerregistry.implico.com"
repository: "devops/test-api"
command: "buildAndPush"
Dockerfile: "TestProject/Dockerfile"
tags: $(API_VERSION)
- job: DeployApi
dependsOn: BuildApi
variables:
API_VERSION: $[ dependencies.DefineVariables.outputs['EnvVariables.API_VERSION'] ]
IS_PROD: $[ dependencies.DefineVariables.outputs['EnvVariables.IS_PROD'] ]
SERVICE_CONNECTION: $[ dependencies.DefineVariables.outputs['EnvVariables.SERVICE_CONNECTION'] ]
SERVICE_NAMESPACE: $[ dependencies.DefineVariables.outputs['EnvVariables.SERVICE_NAMESPACE'] ]
DEPLOYMENT_FILE_API: $[ dependencies.DefineVariables.outputs['EnvVariables.DEPLOYMENT_FILE_API'] ]
steps:
- task: HelmDeploy#0
displayName: "Deploy Api"
inputs:
connectionType: "Kubernetes Service Connection"
kubernetesServiceConnection: $(SERVICE_CONNECTION)
namespace: $(SERVICE_NAMESPACE)
command: "upgrade"
chartType: "FilePath"
chartPath: "devops/deployments/api"
releaseName: "test-api"
overrideValues: image.tag=$(API_VERSION)
valueFile: $(DEPLOYMENT_FILE_API)
and Client:
trigger:
- main
- dev
pool:
name: test-pool
stages:
- stage: Client
jobs:
- job: DefineVariables
displayName: Define Variables
steps:
- task: Bash#3
displayName: Set env variables
name: EnvVariables
inputs:
targetType: "inline"
script: |
PACKAGE_VERSION_CLIENT=$(awk '/version/{gsub(/("|",)/,"",$2);print $2}' ./TestProject/package.json)
if [ $(Build.SourceBranch) == "refs/heads/main" ]
then
IS_PROD=true
CLIENT_VERSION=$PACKAGE_VERSION_CLIENT
SERVICE_CONNECTION=aws-prod
SERVICE_NAMESPACE=test-prod
DEPLOYMENT_FILE_CLIENT=devops/deployments/client/prod-values.yaml
else
IS_PROD=false
CLIENT_VERSION=$PACKAGE_VERSION_CLIENT-dev
SERVICE_CONNECTION=aws-dev
SERVICE_NAMESPACE=test-dev
DEPLOYMENT_FILE_CLIENT="devops/deployments/client/dev-values.yaml"
fi
echo "##vso[task.setvariable variable=IS_PROD;isOutput=true;]$IS_PROD"
echo "##vso[task.setvariable variable=CLIENT_VERSION;isOutput=true;]$CLIENT_VERSION"
echo "##vso[task.setvariable variable=SERVICE_CONNECTION;isOutput=true;]$SERVICE_CONNECTION"
echo "##vso[task.setvariable variable=SERVICE_NAMESPACE;isOutput=true;]$SERVICE_NAMESPACE"
echo "##vso[task.setvariable variable=DEPLOYMENT_FILE_CLIENT;isOutput=true;]$DEPLOYMENT_FILE_CLIENT"
echo Build.Reason: $(Build.Reason)
echo Build.SourceBranch: $(Build.SourceBranch)
echo IS_PROD: $IS_PROD
echo CLIENT_VERSION: $CLIENT_VERSION
echo SERVICE_CONNECTION: $SERVICE_CONNECTION
echo SERVICE_NAMESPACE: $SERVICE_NAMESPACE
echo DEPLOYMENT_FILE_CLIENT: $DEPLOYMENT_FILE_CLIENT
- job: ReadVariable
dependsOn: ["DefineVariables"]
displayName: Read env variables
variables:
CLIENT_VERSION: $[ dependencies.DefineVariables.outputs['EnvVariables.CLIENT_VERSION'] ]
IS_PROD: $[ dependencies.DefineVariables.outputs['EnvVariables.IS_PROD'] ]
SERVICE_CONNECTION: $[ dependencies.DefineVariables.outputs['EnvVariables.SERVICE_CONNECTION'] ]
SERVICE_NAMESPACE: $[ dependencies.DefineVariables.outputs['EnvVariables.SERVICE_NAMESPACE'] ]
DEPLOYMENT_FILE_CLIENT: $[ dependencies.DefineVariables.outputs['EnvVariables.DEPLOYMENT_FILE_CLIENT'] ]
steps:
- task: Bash#3
inputs:
targetType: "inline"
script: |
echo Build.Reason: $(Build.Reason)
echo Build.SourceBranch: $(Build.SourceBranch)
echo IS_PROD: $(IS_PROD)
echo CLIENT_VERSION: $(CLIENT_VERSION)
echo SERVICE_CONNECTION: $(SERVICE_CONNECTION)
echo SERVICE_NAMESPACE: $(SERVICE_NAMESPACE)
echo DEPLOYMENT_FILE_CLIENT: $(DEPLOYMENT_FILE_CLIENT)
- job: BuildClient
displayName: Build Client
dependsOn: ["DefineVariables"]
variables:
CLIENT_VERSION: $[ dependencies.DefineVariables.outputs['EnvVariables.CLIENT_VERSION'] ]
steps:
- task: Docker#2
displayName: "Client - Build and Push Docker Image"
inputs:
containerRegistry: "dockerregistry.implico.com"
repository: "devops/test-client"
command: "buildAndPush"
Dockerfile: "TestProjectClient/Dockerfile"
tags: $(CLIENT_VERSION)
- job: DeployClientDev
displayName: Deploy Client
dependsOn: BuildClient
variables:
CLIENT_VERSION: $[ dependencies.DefineVariables.outputs['EnvVariables.CLIENT_VERSION'] ]
IS_PROD: $[ dependencies.DefineVariables.outputs['EnvVariables.IS_PROD'] ]
SERVICE_CONNECTION: $[ dependencies.DefineVariables.outputs['EnvVariables.SERVICE_CONNECTION'] ]
SERVICE_NAMESPACE: $[ dependencies.DefineVariables.outputs['EnvVariables.SERVICE_NAMESPACE'] ]
DEPLOYMENT_FILE_CLIENT: $[ dependencies.DefineVariables.outputs['EnvVariables.DEPLOYMENT_FILE_CLIENT'] ]
steps:
- task: HelmDeploy#0
displayName: "Deploy Client"
inputs:
connectionType: "Kubernetes Service Connection"
kubernetesServiceConnection: $(SERVICE_CONNECTION)
namespace: $(SERVICE_NAMESPACE)
command: "upgrade"
chartType: "FilePath"
chartPath: "devops/deployments/client"
releaseName: "test-client"
overrideValues: image.tag=$(CLIENT_VERSION)
valueFile: $(DEPLOYMENT_FILE_CLIENT)
For some reason, the variables in the "ReadVariable" and "BuildClient/BuildAPI" jobs are resolved correctly, but not in DeployClient and DeployApi. I don't really understand why. Does anyone have an idea if this is due to the "HelmDeploy#0" and why this is?
Here you can see that the image tag is missing but I can assure you that the other variables are not resolved either.
You need to specify dependencies to pass the variables between the affected jobs. In your scenario please add another dependency dependsOn: DefineVariables for the Deploy jobs:
- job: DeployApi
dependsOn:
- DefineVariables
- BuildApi
API for example:
trigger:
- main
- dev
pool:
name: test-pool
stages:
- stage: Api
jobs:
- job: DefineVariables
displayName: Define Variables
steps:
- task: Bash#3
displayName: Set env variables
name: EnvVariables
inputs:
targetType: "inline"
script: |
PACKAGE_VERSION_API=$(xmlstarlet select --template -value-of /Project/PropertyGroup/Version --nl "TestProject/TestProjectApi.csproj")
if [ $(Build.SourceBranch) == "refs/heads/main" ]
then
IS_PROD=true
API_VERSION=$PACKAGE_VERSION_API
SERVICE_CONNECTION=aws-prod
SERVICE_NAMESPACE=test-prod
DEPLOYMENT_FILE_API=devops/deployments/api/prod-values.yaml
else
IS_PROD=false
API_VERSION=$PACKAGE_VERSION_API-dev
SERVICE_CONNECTION=aws-dev
SERVICE_NAMESPACE=test-dev
DEPLOYMENT_FILE_API=devops/deployments/api/dev-values.yaml
fi
echo "##vso[task.setvariable variable=IS_PROD;isOutput=true;]$IS_PROD"
echo "##vso[task.setvariable variable=API_VERSION;isOutput=true;]$API_VERSION"
echo "##vso[task.setvariable variable=SERVICE_CONNECTION;isOutput=true;]$SERVICE_CONNECTION"
echo "##vso[task.setvariable variable=SERVICE_NAMESPACE;isOutput=true;]$SERVICE_NAMESPACE"
echo "##vso[task.setvariable variable=DEPLOYMENT_FILE_API;isOutput=true;]$DEPLOYMENT_FILE_API"
echo Build.Reason: $(Build.Reason)
echo Build.SourceBranch: $(Build.SourceBranch)
echo IS_PROD: $IS_PROD
echo API_VERSION: $API_VERSION
echo SERVICE_CONNECTION: $SERVICE_CONNECTION
echo SERVICE_NAMESPACE: $SERVICE_NAMESPACE
echo DEPLOYMENT_FILE_API: $DEPLOYMENT_FILE_API
- job: ReadVariable
dependsOn: ["DefineVariables"]
displayName: Read env variables
variables:
API_VERSION: $[ dependencies.DefineVariables.outputs['EnvVariables.API_VERSION'] ]
IS_PROD: $[ dependencies.DefineVariables.outputs['EnvVariables.IS_PROD'] ]
SERVICE_CONNECTION: $[ dependencies.DefineVariables.outputs['EnvVariables.SERVICE_CONNECTION'] ]
SERVICE_NAMESPACE: $[ dependencies.DefineVariables.outputs['EnvVariables.SERVICE_NAMESPACE'] ]
DEPLOYMENT_FILE_API: $[ dependencies.DefineVariables.outputs['EnvVariables.DEPLOYMENT_FILE_API'] ]
steps:
- task: Bash#3
inputs:
targetType: "inline"
script: |
echo Build.Reason: $(Build.Reason)
echo Build.SourceBranch: $(Build.SourceBranch)
echo IS_PROD: $(IS_PROD)
echo API_VERSION: $(API_VERSION)
echo SERVICE_CONNECTION: $(SERVICE_CONNECTION)
echo SERVICE_NAMESPACE: $(SERVICE_NAMESPACE)
echo DEPLOYMENT_FILE_API: $(DEPLOYMENT_FILE_API)
- job: BuildApi
dependsOn: ["DefineVariables"]
variables:
API_VERSION: $[ dependencies.DefineVariables.outputs['EnvVariables.API_VERSION'] ]
steps:
- task: Docker#2
displayName: "Api - Build and Push Docker Image"
inputs:
containerRegistry: "dockerregistry.implico.com"
repository: "devops/test-api"
command: "buildAndPush"
Dockerfile: "TestProject/Dockerfile"
tags: $(API_VERSION)
- job: DeployApi
dependsOn:
- DefineVariables
- BuildApi
variables:
API_VERSION: $[ dependencies.DefineVariables.outputs['EnvVariables.API_VERSION'] ]
IS_PROD: $[ dependencies.DefineVariables.outputs['EnvVariables.IS_PROD'] ]
SERVICE_CONNECTION: $[ dependencies.DefineVariables.outputs['EnvVariables.SERVICE_CONNECTION'] ]
SERVICE_NAMESPACE: $[ dependencies.DefineVariables.outputs['EnvVariables.SERVICE_NAMESPACE'] ]
DEPLOYMENT_FILE_API: $[ dependencies.DefineVariables.outputs['EnvVariables.DEPLOYMENT_FILE_API'] ]
steps:
- task: HelmDeploy#0
displayName: "Deploy Api"
inputs:
connectionType: "Kubernetes Service Connection"
kubernetesServiceConnection: $(SERVICE_CONNECTION)
namespace: $(SERVICE_NAMESPACE)
command: "upgrade"
chartType: "FilePath"
chartPath: "devops/deployments/api"
releaseName: "test-api"
overrideValues: image.tag=$(API_VERSION)
valueFile: $(DEPLOYMENT_FILE_API)
Related
I'm trying to allow for a single variable that can specify the build id or branch to deploy from. This means I need to type coerce, and it's failing if I try to call a ge (greater than or equal) or lt (less than) on a string. It looks like I might have to use bash commands to do the type conversion for me, so I was wondering if anyone had this handy.
Here's the steps I'm using:
- ${{ each deployment in parameters.deployments }}:
- deployment: Deploy_${{deployment.serviceName}}_${{ parameters.region }}
pool:
vmImage: ${{ parameters.vmImage }}
displayName: 'Deploy ${{ deployment.serviceName }} ${{ parameters.region }}'
${{ if not(eq(parameters.kubernetesServiceEndpoint, '')) }}:
environment: ${{ parameters.kubernetesServiceEndpoint }}
${{ elseif not(and(eq(parameters.azureResourceGroup, ''), eq(parameters.kubernetesCluster, ''))) }}:
environment: ${{ parameters.environment }}
strategy:
runOnce:
deploy:
steps:
# Disable the automatic downloading of artifacts, because we will specifying exactly what we need
- download: none
- task: DownloadPipelineArtifact#2
condition: ${{ ge(0, deployment.branchBuildId) }}
inputs:
source: 'specific'
project: ${{ deployment.project }}
pipeline: ${{ deployment.pipeline }}
runVersion: 'latestFromBranch'
runBranch: ${{ deployment.branchBuildId }}
patterns: 'DeploymentData/*'
displayName: 'Download Latest Artifacts'
- task: DownloadPipelineArtifact#2
condition: ${{ lt(0, deployment.branchBuildId) }}
inputs:
source: 'specific'
project: ${{ deployment.project }}
pipeline: ${{ deployment.pipeline }}
runVersion: 'specific'
runId: ${{ deployment.branchBuildId }}
#runBranch: ${{ deployment.branch }}
patterns: 'DeploymentData/*'
displayName: 'Download Specific Artifacts (${{ deployment.buildId }})'
I want these to be mutually exclusive (run the first if pulling the latest from a specific branch, and run the second if downloading from a specific build id). By using a single parameter I can avoid another control parameter to decide which version should be used.
Two things:
1, If you use script to handle, then 'compile time usage' is unable to use.
You need to use runtime usage:
variables['xxx']
2, Variables only support passing string, you can compare them and then use logging command to output the result:
trigger:
- none
pool:
vmImage: ubuntu-latest
parameters:
- name: str1
default: 110
- name: str2
default: 130
steps:
- task: PowerShell#2
name: outputresult
inputs:
targetType: 'inline'
script: |
$str1 = "${{parameters.str1}}"
$str2 = "${{parameters.str2}}"
Write-Host $str1
Write-Host $str2
# powershell compare the strings greater than or less than
if ($str1 -gt $str2) {
$result = "greater than"
Write-Host $result
Write-Host "##vso[task.setvariable variable=result;isoutput=true]$result"
} elseif ($str1 -lt $str2) {
$result = "less than"
Write-Host $result
Write-Host "##vso[task.setvariable variable=result;isoutput=true]$result"
} else {
$result = "equal"
Write-Host $result
Write-Host "##vso[task.setvariable variable=result;isoutput=true]$result"
}
#Below is the handling logic
- task: PowerShell#2
condition: eq('greater than',variables['outputresult.result'])
inputs:
targetType: 'inline'
script: |
# Write your PowerShell commands here.
Write-Host "greater than"
- task: PowerShell#2
condition: eq('less than',variables['outputresult.result'])
inputs:
targetType: 'inline'
script: |
# Write your PowerShell commands here.
Write-Host "less than"
Result:
In my CI pipeline I am trying to publish message to service bus and its working when its just some hardcoded text or variables, here Using "PublishToAzureServiceBus" task .
But problem is when trying to a read file from repository and then publish that to service bus.
I have tried using read file using scripting language and put to variable but its not able to work as variable is not storing big json file.
Is there any way to read file directly when publishing message to service bus.
Below is sample code snippet for debugging
trigger:
- none
pool:
vmImage: ubuntu-latest
parameters:
- name: ProjectName
displayName: Project Name
type: string
default: DevOpsDemo
- name: repoName
displayName: repo Name
type: string
default: ProjectCode
- name: branchRef
displayName: Branch Name
type: string
default: main
variables:
- name: jobStatus
value: "Failed"
- name: projectFile
value: ""
stages:
- stage: Stage1
displayName: Stage 1
jobs:
- job: CheckOutRepo
displayName: CheckOut-Repo Display
steps:
- script: |
echo "Checkout for " ${{ parameters.ProjectName}} : ${{ parameters.repoName}} : ${{ parameters.branchRef}}
name: PrintMessage
- checkout: git://${{ parameters.ProjectName}}/${{ parameters.repoName}}#refs/heads/${{ parameters.branchRef}}
name: Checkout
- task: PythonScript#0
inputs:
scriptSource: 'inline'
script: |
import json
import requests
f = open('project-release.json')
projectFile = json.load(f)
print(projectFile)
f.close()
print("Afterclosing")
print(projectFile)
- script: |
echo "Project release file" $(cat project-release.json)
name: TestPrint
- task: CopyFiles#2
inputs:
SourceFolder: 'services'
Contents: '**'
TargetFolder: $(Build.ArtifactStagingDirectory)
name: CopyFiles
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: $(Build.ArtifactStagingDirectory)
ArtifactName: 'drop'
publishLocation: 'Container'
name: PublishArtifacts
- bash: |
echo "##vso[task.setvariable variable=jobStatus]Success"
name: setVar
- bash: |
echo "##vso[task.setvariable variable=jobStatus;isOutput=true]$(jobStatus)"
echo "##vso[task.setvariable variable=projectFile;isOutput=true]$(cat project-release.json)"
name: SetStatus
condition: always()
- stage: Stage2
displayName: Stage 2
condition: always()
jobs:
- job: Publish
pool: server
variables:
jobStatus: $[ stageDependencies.Stage1.CheckOutRepo.outputs['SetStatus.jobStatus'] ]
projectFile: $[ stageDependencies.Stage1.CheckOutRepo.outputs['SetStatus.projectFile'] ]
steps:
- task: PublishToAzureServiceBus#1
inputs:
azureSubscription: 'SBConnection'
messageBody: |
{
"Status": "$(jobStatus)",
"BuildID": "$(build.buildid)",
"BuildNumber":"$(build.buildnumber)",
"projectFile":$(cat project-release.json)
}
signPayload: false
waitForCompletion: false
condition: always()
I am able to solve this by using setvariable in bash script as below
pool:
vmImage: ubuntu-latest
stages:
- stage: Stage1
displayName: Stage 1
jobs:
- job: CheckOutRepo
displayName: CheckOut-Repo Display
steps:
- checkout: git://${{ parameters.ProjectName}}/${{ parameters.repoName}}#refs/heads/${{ parameters.branchRef}}
name: Checkout
- bash: |
data=$(cat project-release.json)
echo "##vso[task.setvariable variable=jobStatus;isOutput=true]$(jobStatus)"
echo "##vso[task.setvariable variable=data;isOutput=true]"$data
name: SetStatus
condition: always()
- stage: Stage2
displayName: Stage 2
condition: always()
jobs:
- job: Publish
pool: server
variables:
jobStatus: $[ stageDependencies.Stage1.CheckOutRepo.outputs['SetStatus.jobStatus'] ]
projectFile: $[ stageDependencies.Stage1.CheckOutRepo.outputs['SetStatus.data'] ]
steps:
- task: PublishToAzureServiceBus#1
inputs:
azureSubscription: 'SBConnection'
messageBody: |
{
"Status": "$(jobStatus)",
"BuildID": "$(build.buildid)",
"BuildNumber":"$(build.buildnumber)",
"projectFile":$(projectFile)
}
signPayload: false
waitForCompletion: false
condition: always()
EDIT
I have got a pipeline below and would want it to run an inline script based on the time of the day.
The question should have been around pipelines rather than ARM template.
schedules:
- cron: "0 10 * * *"
displayName: Test 1
branches:
include:
- master
always: true
- cron: "0 21 * * *"
displayName: Test 2
branches:
include:
- master
always: true
steps:
- ${{ if eq(schedules.cron, '0 10 ***') }}:
- task: AzureCLI#2
name: RunProcess
displayName: Run test 1
inputs:
azureSubscription: serviceConnection
scriptLocation: 'inlineScript'
scriptType: bash
failOnStandardError: true
inlineScript: |
echo 'starting process 1'
The way I was able to do this can be found below.
steps:
- task: PowerShell#02
name: taskname
displayName: task display name
inputs:
azureSubscription: $(subnamee)
scriptLocation: 'inlineScript'
failOnStandardError: true
targetType: 'inline'
script: |
$h = (Get-Date).hour
if ($h -eq 10)
{
echo 'command 1'
}
if ($h -eq 21)
{
echo 'command 2'
}
I was able to do this by changing the script type to PS.
The way I was able to do this can be found below.
steps:
- task: PowerShell#02
name: taskname
displayName: task display name
inputs:
azureSubscription: $(subnamee)
scriptLocation: 'inlineScript'
failOnStandardError: true
targetType: 'inline'
script: |
$h = (Get-Date).hour
if ($h -eq 10)
{
echo 'command 1'
}
if ($h -eq 21)
{
echo 'command 2'
}
I want a stage in an Azure DevOps pipeline to be executed depending on the content of a variable set in a previous stage.
Here is my pipeline:
stages:
- stage: plan_dev
jobs:
- job: terraform_plan_dev
steps:
- bash: echo '##vso[task.setvariable variable=terraform_plan_exitcode;isOutput=true]2'
name: terraform_plan
- stage: apply_dev
dependsOn: plan_dev
condition: eq(stageDependencies.plan_dev.terraform_plan_dev.outputs['terraform_plan.terraform_plan_exitcode'], '2')
jobs:
- deployment: "apply_dev"
...
The idea is to skip the apply_dev stage, if the plan_dev stage shows no changes. Background is that we have manual approval for the deployment in the plan_dev stage that we want to skip if there are no changes to be approved.
Unfortunately this doesn't seem to work. No matter whether the variable terraform_plan_exitcode is set with the expected value (2) or not, the apply_dev stage is skipped.
For the syntax, I followed the documentation here that says:
stageDependencies.StageName.JobName.outputs['StepName.VariableName']
I have seen this same issue. You need to use the dependencies variable instead of the stageDependencies:
stages:
- stage: plan_dev
jobs:
- job: terraform_plan_dev
steps:
- bash: echo '##vso[task.setvariable variable=terraform_plan_exitcode;isOutput=true]2'
name: terraform_plan
- stage: apply_dev
dependsOn: plan_dev
condition: eq(dependencies.plan_dev.outputs['terraform_plan_dev.terraform_plan.terraform_plan_exitcode'], '2')
jobs:
- deployment: "apply_dev"
The following is a more complete example of something I have working with Terraform Plan + conditional Apply:
stages:
- stage: Build_zip_plan
displayName: Build portal, zip files and terraform plan
jobs:
- job: Build_portal_zip_files_terraform_plan
pool:
vmImage: 'ubuntu-latest'
steps:
- task: Cache#2
displayName: 'Register TF cache'
inputs:
key: terraform | $(Agent.OS) | $(Build.BuildNumber) | $(Build.BuildId) | $(Build.SourceVersion) | $(prefix)
path: ${{ parameters.tfExecutionDir }}
- task: TerraformInstaller#0
displayName: 'Install Terraform'
inputs:
terraformVersion: ${{ parameters.tfVersion }}
- task: TerraformTaskV1#0
displayName: 'Terraform Init'
inputs:
provider: 'azurerm'
command: 'init'
workingDirectory: ${{ parameters.tfExecutionDir }}
backendServiceArm: ${{ parameters.tfStateServiceConnection }}
backendAzureRmResourceGroupName: ${{ parameters.tfStateResourceGroup }}
backendAzureRmStorageAccountName: ${{ parameters.tfStateStorageAccount }}
backendAzureRmContainerName: ${{ parameters.tfStateStorageContainer }}
backendAzureRmKey: '$(prefix)-$(environment).tfstate'
- task: TerraformTaskV1#0
displayName: 'Terraform Plan'
inputs:
provider: 'azurerm'
command: 'plan'
commandOptions: '-input=false -out=deployment.tfplan -var="environment=$(environment)" -var="prefix=$(prefix)" -var="tenant=$(tenant)" -var="servicenow={username=\"$(servicenowusername)\",instance=\"$(servicenowinstance)\",password=\"$(servicenowpassword)\",assignmentgroup=\"$(servicenowassignmentgroup)\",company=\"$(servicenowcompany)\"}" -var="clientid=$(clientid)" -var="username=$(username)" -var="password=$(password)" -var="clientsecret=$(clientsecret)" -var="mcasapitoken=$(mcasapitoken)" -var="portaltenantid=$(portaltenantid)" -var="portalclientid=$(portalclientid)" -var="customerdisplayname=$(customerdisplayname)" -var="reportonlymode=$(reportonlymode)"'
workingDirectory: ${{ parameters.tfExecutionDir }}
environmentServiceNameAzureRM: ${{ parameters.tfServiceConnection }}
- task: PowerShell#2
displayName: 'Check Terraform plan'
name: "Check_Terraform_Plan"
inputs:
filePath: '$(Build.SourcesDirectory)/Pipelines/Invoke-CheckTerraformPlan.ps1'
arguments: '-TfPlan ''${{ parameters.tfExecutionDir }}/deployment.tfplan'''
pwsh: true
- stage:
dependsOn: Build_zip_plan
displayName: Terraform apply
condition: eq(dependencies.Build_zip_plan.outputs['Build_portal_zip_files_terraform_plan.Check_Terraform_Plan.TFChangesPending'], 'yes')
jobs:
- deployment: DeployHub
displayName: Apply
pool:
vmImage: 'ubuntu-latest'
environment: '$(prefix)'
strategy:
runOnce:
deploy:
steps:
- checkout: self
- task: Cache#2
displayName: 'Get Cache for TF Artifact'
inputs:
key: terraform | $(Agent.OS) | $(Build.BuildNumber) | $(Build.BuildId) | $(Build.SourceVersion) | $(prefix)
path: ${{ parameters.tfExecutionDir }}
- task: TerraformInstaller#0
displayName: 'Install Terraform'
inputs:
terraformVersion: ${{ parameters.tfVersion }}
- task: TerraformTaskV1#0
displayName: 'Terraform Apply'
inputs:
provider: 'azurerm'
command: 'apply'
commandOptions: 'deployment.tfplan'
workingDirectory: ${{ parameters.tfExecutionDir }}
environmentServiceNameAzureRM: ${{ parameters.tfServiceConnection }}
#Marius is correct. So this works
stages:
- stage: plan_dev
jobs:
- job: terraform_plan_dev
steps:
- bash: echo '##vso[task.setvariable variable=terraform_plan_exitcode;isOutput=true]2'
name: terraform_plan
- stage: apply_dev
dependsOn: plan_dev
variables:
varFromA: $[ stageDependencies.plan_dev.terraform_plan_dev.outputs['terraform_plan.terraform_plan_exitcode'] ]
condition: eq(dependencies.plan_dev.outputs['terraform_plan_dev.terraform_plan.terraform_plan_exitcode'], 2)
jobs:
- job: apply_dev
steps:
- bash: echo 'apply $(varFromA)'
name: terraform_apply
When you refer stage to stage dependencies you have different syntax
"dependencies": {
"<STAGE_NAME>" : {
"result": "Succeeded|SucceededWithIssues|Skipped|Failed|Canceled",
"outputs": {
"jobName.stepName.variableName": "value"
}
},
"...": {
// another stage
}
}
And when you refer to job to job across stage you have different syntax
"stageDependencies": {
"<STAGE_NAME>" : {
"<JOB_NAME>": {
"result": "Succeeded|SucceededWithIssues|Skipped|Failed|Canceled",
"outputs": {
"stepName.variableName": "value"
}
},
"...": {
// another job
}
},
"...": {
// another stage
}
}
What is funny when you have job to job in one stage we use dependecies syntax again
"dependencies": {
"<JOB_NAME>": {
"result": "Succeeded|SucceededWithIssues|Skipped|Failed|Canceled",
"outputs": {
"stepName.variableName": "value1"
}
},
"...": {
// another job
}
}
This is a bit confusing and consider this in this as
when you are on some level stage, job and refer to the same level from job to job or from stage to stage you have dependencies syntax
when you want to refer from deeper level like from job to stage you should use stageDependencies
What is funny, in above example I used this on stage level:
variables:
varFromA: $[ stageDependencies.plan_dev.terraform_plan_dev.outputs['terraform_plan.terraform_plan_exitcode'] ]
but this is evaluated at runtime and is evaluated from the job, so it is correct and is evaluated correctly.
I hope it added a value to previous answer.
TerraformTaskV2 has changesPresent output variable now, which can be used to skip apply stage.
add name: to the plan task
stages:
- stage: terraform_plan_STAGE
jobs:
- job: plan_JOB
...
steps:
...
- task: TerraformTaskV2#2
name: 'plan_TASK' # <===========
displayName: 'plan'
inputs:
provider: 'azurerm'
command: 'plan'
...
add condition: to apply stage and check if changesPresent is true
- stage: terraform_apply
dependsOn: [terraform_plan]
condition: eq(dependencies.terraform_plan_STAGE.outputs['plan_JOB.plan_TASK.changesPresent'], 'true')
reference:
https://github.com/microsoft/azure-pipelines-terraform/tree/main/Tasks/TerraformTask/TerraformTaskV2#output-variables
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops&tabs=yaml%2Cbatch#use-outputs-in-a-different-stage
I'm trying to debug a pipeline script in Azure to see what the variables are, but I'm not sure of the syntax.
I have looked through the documentation
What I have tried so far
parameters:
deploymentName: ""
dependsOn: ""
env: ""
dockerfilePath: ""
buildContext: ""
repository: ""
envGroup: ""
dockerRegistryServiceConnection: ""
tag: ""
token: ""
runTests: ""
jobs:
- deployment: ${{ parameters.deploymentName }}
dependsOn: ${{ parameters.dependsOn }}
pool: "Private Docker"
environment: "${{ parameters.envGroup }}"
strategy:
runOnce:
deploy:
steps:
- checkout: self
- task: Docker#2
displayName: "ACR Login"
inputs:
command: login
containerRegistry: "${{ parameters.dockerRegistryServiceConnection }}"
##[debug]Debug!!!!!1 $runTests
##[debug]Debug!!!!!2 $parameters.runTests
##[debug]Debug!!!!!3 ${{parameters.runTests}}
- script: echo runTests in dockerbuild.yml ${{parameters.runTests}}
- script: echo runTests in dockerbuild.yml $parameters.runTests
- script: echo runTests in dockerbuild.yml $runTests
- task: Docker#2
displayName: Build
inputs:
command: build
repository: "${{ parameters.repository }}"
buildContext: "${{ parameters.buildContext }}"
dockerfile: "${{ parameters.dockerfilePath }}"
arguments: "--build-arg TOKEN=${{ parameters.token }} --build-arg RUNTESTS=${{ parameters.runTests }}"
tags: |
$(Build.BuildId)
latest
Problem
The debug and echo do not print anything in the azure pipeline build:
##[debug]Debug!!!!!1 $runTests
##[debug]Debug!!!!!2 $parameters.runTests
##[debug]Debug!!!!!3 ${{parameters.runTests}}
- script: echo runTests in dockerbuild.yml ${{parameters.runTests}}
- script: echo runTests in dockerbuild.yml $parameters.runTests
- script: echo runTests in dockerbuild.yml $runTests
Question
Is this the correct syntax for printing to the azure bash when running the yaml file?
Debugging yaml pipeline script
It seems that you did not define the parameters correctly. We need provide the name, displayName, type, default, values like following (Not every attribute is required):
parameters:
- name: deploymentName
type: string
default: Product
Then we could use the ${{ parameters.deploymentName }} to get the value.
Please check this document Runtime parameters for some more details.
My test YAML file:
parameters:
- name: deploymentName
type: string
default: Product
- name: dependsOn
type: string
default: Test
- name: envGroup
type: string
default: test
- name: runTests
type: number
default: 123
stages:
- stage: Build
jobs:
- job: Build
displayName: Build
pool:
name: MyPrivateAgent
steps:
- script: echo This is for Build
- stage: Dev
jobs:
- job: Test
displayName: Test
pool:
name: MyPrivateAgent
steps:
- script: echo This is for test
- deployment: ${{ parameters.deploymentName }}
dependsOn: ${{ parameters.dependsOn }}
pool:
name: MyPrivateAgent
environment: "${{ parameters.envGroup }}"
strategy:
runOnce:
deploy:
steps:
- checkout: self
- script: echo runTests in dockerbuild.yml ${{parameters.runTests}}
The test result:
Try this:
${{ parameters.runTests}}