I need to setup a pipeline that has 2 defined schedules, one to execute script x in job 1, and one to execute script y in job 2.
Ill try and show in the snippet what I need as well:
schedules:
- cron: '0 16 * * *'
displayName: 'schedule 1'
branches:
include:
- master
always: True
- cron: '0 4 * * *'
displayName: 'schedule 2'
branches:
include:
- master
always: True
jobs:
- job: job x
condition: eq( variables.schedule, 'schedule 1')
...
- job: job y
condition: eq( variables.schedule, 'schedule 2')
I defined both schedules, they run correctly, however I cant seem to figure out how to have the jobs 'condition' comply with the current run schedule.
I cannot find anything about pre defined variables saying anything about the schedule.
Related
https://medium.com/tech-start/looping-a-task-in-azure-devops-ac07a68a5556
Below is my Main Pipeline:
pool:
vmImage: 'windows-latest'
parameters:
- name: environment
displayName: 'Select Environment To Deploy'
type: string
values:
- A1
- A2
default: A1
variables:
- name: abc
value: 'firm20 firm201' # How to declare Array variable?
stages:
- stage: stage01
displayName: 'Deploy in Environment '
jobs:
- template: templates/test_task_for.yml # Template reference
# parameters:
# list: $(abc)
Now is my template pipeline below:
parameters:
- name: list
type: ??? #What type here pls?
default: [] #? is this correct?
jobs:
- job: connectxyz
displayName: 'Connect'
steps:
- ${{each mc in variables.abc}}: # parameters.list
- task: CmdLine#2
inputs:
script: 'echo Write your commands here. ${{mc}}'
I need to run CmdLine#2 task multiple times
How to define values in Main Pipeline abc so that it can be passed as parameter to Template file?
What datatype of Parameter so it can work as array?
Azure Pipelines does not have an array parameter type. You can use an object type and iterate over it.
User-defined variables are not typed. A variable is a string, period. You cannot define a variable containing an array of items.
However, you could add a delimited list, then split the list.
i.e.
variables:
foo: a,b,c,d
- ${{ each x in split(variables.foo, ',') }}:
- script: echo ${{ x }}
I'm trying to setup a bot that takes three inputs, two from buttons, and one from text. Currently, I'm able to pass in all of the required input. However, when it comes time for the final action, the bot does not seem to know what to do, and returns either a greeting or a message that it does not understand me. Is there anything wrong with my rules or domain files?
rule.yml
version: "2.0"
rules:
- rule: Activate conv_get_job_status
steps:
- intent: get_job_query
- action: action_populate_user_bot_info
- action: job_type_selection #this creates a list of buttons for job type selection
- action: job_type_input #job type form, to set job_type slot
- active_loop: job_type_input
- rule: Submit conv_get_job_status
condition:
- active_loop: job_type_input
- active_loop: job_name_and_number_input
steps:
- action: job_type_input
- active_loop: null
- slot_was_set:
- requested_slot: null
- action: job_name_selection #this creates a list of buttons for job name selection
- action: job_name_and_number_input #job name and number form, to set job_name and job number slots
- active_loop: job_name_and_number_input
- action: job_name_and_number_input
- active_loop: null
- slot_was_set:
- requested_slot: null
- action: action_return_job_status #various utterances, currently not called.
testjob_domain.yml
version: '2.0'
entities:
- job_type
- job_name
- job_number
intents:
- get_job_query
actions:
- action_reset_job_type_input
- job_number_query
- utter_ask_job_number
- utter_ask_job_name
- utter_ask_job_type
- action_return_job_status
- job_name_selection
- get_job_status
- action_reset_get_job_status
- job_type_selection
- action_reset_job_name_and_number_input
responses:
utter_ask_job_type:
- text: Select job type
utter_ask_job_name:
- text: Select job name
utter_ask_job_number:
- text: Enter job number
slots:
execute_action:
type: text
job_type:
type: text
job_name:
type: text
job_number:
type: text
forms:
job_type_input:
job_type:
- type: from_text
intent_name: None
job_name_and_number_input:
job_name:
- type: from_text
intent_name: None
job_number:
- type: from_text
intent_name: None
The code for action_return_job_status:
from .action_utils import *
class ActionReturnJobStatus(Action):
def name(self):
return "action_return_job_status"
def run(self, dispatcher, tracker, domain):
logger.info("Action - action_return_job_status")
logger.info("execute_action_slot: "+str(tracker.slots['execute_action']))
record_bot_event(tracker, dispatcher, 'new_session', auth_critical=authorization_critical)
# set auth_critical=True to skip action code execution
if not verifyBAMToken(tracker, dispatcher, auth_critical=authorization_critical):
return
logger.info("RUNNING GET ACTION")#I would expect to see either this in the logs or the utterances. Neither happens
dispatcher.utter_message("RUNNING GET ACTION")
return [SlotSet('latest_faq_question', None), SlotSet('latest_application_name', None)]
That's my matrix:
jobs:
check-test:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.6", "3.7", "3.8", "3.9", "3.10"]
And I have this current conditional (that's working):
if: matrix.python-version == 3.6
How can I replace 3.6 by the first item of my matrix? I mean, how to tell in yaml:
if: matrix.python-version == "the_first_item_from_matrix.python-version"
And no, matrix.python-version[0] (or [1], I don't know how it's indexed) won't work.
The reasoning here is: in some point I will drop 3.6 support and I don't want to remember to remove any 3.6 hardcoded in my workflow.
I propose a simple workaround using the strategy.job-index property. According to the documentation, this property is defined as follows:
The index of the current job in the matrix. Note: This number is a zero-based number. The first job's index in the matrix is 0.
You can use this property to ensure that a step is only executed for the first job in the matrix.
Here is a minimal workflow to demonstrate the idea:
name: demo
on:
push:
jobs:
demo:
strategy:
fail-fast: false
matrix:
python-version: ['3.7', '3.10']
runs-on: ubuntu-latest
steps:
- if: strategy.job-index == 0
run: echo '3.7' && exit 1
- if: strategy.job-index == 1
run: echo '3.10' && exit 1
Here are screenshots to show the result:
Sadly this workaround will get more complicated once we start to work with a matrix that has more than one dimension.
In azure DevOps YML pipelines, is it possible to trigger a pipeline A stage1 from pipeline B stageY ?
There's a Trigger Build custom task that you can use. It triggers the whole pipeline, but you can use stage conditions to skip stages that should not run.
# in pipeline A
- task: TriggerBuild#3
displayName: 'Trigger a new build pipelineB'
inputs:
buildDefinition: 'pipelineB'
waitForQueuedBuildsToFinish: true
waitForQueuedBuildsToFinishRefreshTime: 10
buildParameters: 'stageY: true, stageX: false, stageZ: false'
authenticationMethod: 'OAuth Token'
password: '$(System.AccessToken)'
# in pipeline B
variables: [] # do not define stageX, stageY, stageZ variables here, or it won't work
stages:
- stage: stageX
condition: ne(variables.stageX, false)
...
- stage: stageY
condition: ne(variables.stageY, false)
...
- stage: stageZ
condition: ne(variables.stageZ, false)
...
I have a pool of self hosted VMs (MyTestPool) half of which is dedicated to installing & testing a 'ON' build (that has few features turned on) and a 'OFF' build (that's a default installation). All my test agent VMs have 'ALL' as a user defined capability. Half of them are also tagged 'ON', and the other half 'OFF'.
Now I have 2 stages called DEPLOYOFF & DEPLOYON that can be skipped if a Variable Group variable 'skipDeployOffStage' or 'skipDeployOnStage' is set to true. What I would like to do is to use 'ALL' as an agent demand if only ON / OFF is to be tested. If both ON & OFF are to be tested, the appropriate stage would demand their respective 'ON' or 'OFF' VMs.
Question: The ${{ if }} DOES NOT WORK.
trigger: none
pr: none
pool: 'MyBuildPool'
variables:
- group: TEST_IF_VARIABLE_GROUP
- name: subPool
value: 'ON'
- name: useAllPool
value: $[ or(eq( variables.skipDeployOnStage, true), eq( variables.skipDeployOffStage, true)) ]
stages:
- stage: DEPLOYOFF
condition: ne(variables['skipDeployOffStage'], 'true')
variables:
# The test stage's subpool
${{ if variables.useAllPool }}:
subPool: 'ALL'
${{ if not(variables.useAllPool) }}:
subPool: 'OFF'
pool:
name: 'MyTestPool'
demands:
- ${{ variables.subPool }}
jobs:
- job:
steps:
- checkout: none
- pwsh: |
Write-Host ("Value of useAllPool is: {0}" -f '$(useAllPool)')
Write-Host ("Value of VG variable 'skipDeployOnStage' is {0} and 'skipDeployOffStage' is {1}" -f '$(skipDeployOnStage)', '$(skipDeployOffStage)')
Write-Host ("Subpool is {0}" -f '$(subPool)')
displayName: 'Determined SubPool'
The Output when one of the flags is false:
2020-08-02T18:39:05.5849160Z Value of useAllPool is: True
2020-08-02T18:39:05.5854283Z Value of VG variable 'skipDeployOnStage' is true and 'skipDeployOffStage' is false
2020-08-02T18:39:05.5868711Z Subpool is ALL
The Output when both are false:
2020-08-02T18:56:40.5371875Z Value of useAllPool is: False
2020-08-02T18:56:40.5383258Z Value of VG variable 'skipDeployOnStage' is false and 'skipDeployOffStage' is false
2020-08-02T18:56:40.5386626Z Subpool is ALL
What am I missing?
There are two issues that cause your code to run incorrectly:
1.The ${{if}}:
The way you write ${{if}} is incorrect, and the correct script is:
${{ if eq(variables['useAllPool'], true)}}:
subPool: 'ALL'
${{ if ne(variables['useAllPool'], true)}}:
subPool: 'OFF'
2.The definition of variables.useAllPool:
You use a runtime expression ($[ <expression> ]), so when the ${{if}} is running, the value of variables.useAllPool is '$[ or(eq( variables.skipDeployOnStage, true), eq( variables.skipDeployOffStage, true)) ]' instead of true or false.
To solve this issue, you need to use compile time expression ${{ <expression> }}.
However, when using compile time expression, it cannot contain variables from variable groups. So you need to move the variables skipDeployOnStage and skipDeployOffStage from variable group to YAML file.
So, you can solve the issue by the following steps:
1.Delete the variables skipDeployOnStage and skipDeployOffStage from the variable group TEST_IF_VARIABLE_GROUP.
2.Modify the YAML file:
trigger: none
pr: none
pool: 'MyBuildPool'
variables:
- group: TEST_IF_VARIABLE_GROUP
- name: subPool
value: 'ON'
- name: skipDeployOnStage
value: true
- name: skipDeployOffStage
value: false
- name: useAllPool
value: ${{ or(eq( variables.skipDeployOnStage, true), eq( variables.skipDeployOffStage, true)) }}
stages:
- stage: DEPLOYOFF
condition: ne(variables['skipDeployOffStage'], 'true')
variables:
# The test stage's subpool
${{ if eq(variables['useAllPool'], true)}}:
subPool: 'ALL'
${{ if ne(variables['useAllPool'], true)}}:
subPool: 'OFF'
pool:
name: 'MyTestPool'
demands:
- ${{ variables.subPool }}
jobs:
- job:
steps:
- checkout: none
- pwsh: |
Write-Host ("Value of useAllPool is: {0}" -f '$(useAllPool)')
Write-Host ("Value of VG variable 'skipDeployOnStage' is {0} and 'skipDeployOffStage' is {1}" -f '$(skipDeployOnStage)', '$(skipDeployOffStage)')
Write-Host ("Subpool is {0}" -f '$(subPool)')
displayName: 'Determined SubPool'
You can modify the value of skipDeployOnStage and skipDeployOffStage in YAML file to test whether this solution works.
My requirement:
Have a self hosted VM pool of say, 20 machines
10 of them have "OFF" as a User Capability (Name as OFF & value as blank) 10 of them have
"ON" as a User capability (Name as ON & value as blank)
All of them have a "ALL" as an additional user capability
The pipeline basically installs 2 variations of the products and runs tests on them on the designated machines in the pool (ON / OFF)
When the user wants to run both OFF & ON deployments, I have stages that demand OFF or ON run
What I want to do is when the user wants only ON or OFF deployment, to save time I want to use all 20 of the machines, deploy and test so I can reduce overall testing time.
I was trying vainly, to replace the pool demand at run time as I did not want to update user capabilities for 20-50 VMs just prior to the run every time. That's what I have to do if use the traditional demand syntax:
pool:
name: 'MyTestPool'
demands:
# There's no OR or other syntax supported here
# LVALUE must be a built-in variable such as Agent.Name OR a User capability
# I will have to manually change the DeploymentType before the pipeline is run
- DeploymentType -equals $(subPool)
So, I was trying to ascertain the value of the subPool at run time and use the below syntax so I don't have to manually configure the user capabilities before run.
pool:
name: ${{ parameters.pool }}
demands:
- ${{ parameters.subPool}}
Here's my research:
You can definitely mix compile & run time expressions. The compile time expression evaluates to whatever the value was at the compile time. If it's an expression or a variable (and not a constant), the entire expression or variable is substituted as Jane Ma-MSFT notes.
For example, I have been able to use queue time variables in compile time expressions without issues. For example, I've used which pool to use as a queue time variable and then pass on the same to a template which uses a compile time syntax for the value.
parameters:
- name: pool
type: string
jobs:
- job: ExecAllPrerequisitesJob
displayName: 'Run Stage prerequisites one time from a Single agent'
pool:
name: ${{ parameters.pool }}
However, the real issue is where you're using the compile time expression. Essentially, in the above, the entire ${{ parameters.pool }} gets replaced by $(buildPool), a queue time variable at compile time. But, pool supports using a variable for the pool name. This is so convoluted and undocumented where you can use expressions, variables (compile or run) and where you must use constants.
One such example:
jobs:
- job: SliceItFourWays
strategy:
parallel: 4 # $[ variables['noOfVMs'] ] is the ONLY syntax that works to replace 4
In some places, such as Pool demands, Microsoft's YAML parser dutifully replaces the variable. However, the run time doesn't support custom variables as LVALUE.
It only supports evaluation of custom run time variables in the RVALUE portion of the demand.
pool:
name: ${{ parameters.buildPool }} # Run time supports variables here
demands:
- ${{ parameters.subPool }} # LVALUE: Must be resolved to a CONSTANT at compile time
- poolToRunOn -equals '${{ parameters.subPool }}' # RVALUE... custom user capability. This evaluates and applies the demand correctly.
Conclusion:
MICROSOFT'S YAML IMPLEMENTATION SUCKS!