We have one pipeline and one only (we cannot and do not want to create a 2nd pipeline or do it with a separate pipeline, it has to be done in the same pipeline), that pipeline has a task to either stop or start a function while accepting a variable group (its required) from the library (we specify those store A-E variables on the YAML but they also exist in the library), and specify the Azure subscription. Currently, we run this pipeline manually, this is what it looks like before I run it
What I'm looking for is a feature to automate this pipeline to run at 7 PM CST with the start function as the task, accept a variable group, and specify which azure subscription that i want. Then, at 6 AM CST the next day, I need to have that SAME pipeline to run a build with stop function as the task, accept a variable group, and specify which azure subscription i want.
I found a scheduler feature in the CI pipeline but it doesnt allow me to specify which variable group I want from the library, no option to select either start or stop the function, and no option to select the subscription. This is what I'm expecting to see
If it any helps this is the .YAML code that i have (some stuff has been removed for privacy purposes)
trigger:
- none
pool:
vmImage: 'windows-latest'
parameters:
- name: variableGroup
displayName: Variable Group
type: string
values:
- 'variable for store A'
- 'variable for store B'
- 'variable for store C'
- 'variable for store D'
- 'variable for store E'
- name: artifactVersion
displayName: ArtifactVersion (* (latest) or 1.{sprintNumber}.{ReleaseNo})
type: string
default: '*'
- name: Function
displayName: Function
type: string
default: 'deploy'
values:
- deploy
- name: task
displayName: ExecuteTask
type: string
default: ''
values:
- start thefunction
- stop the function
- name: Subscription
displayName: Subscription
type: string
values:
- 'sandbox'
- 'production '
I am afraid that there is no such method can meet your requirements for the time being.
Refer to this doc: Scheduled triggers
schedules:
- cron: string # cron syntax defining a schedule
displayName: string # friendly name given to a specific schedule
branches:
include: [ string ] # which branches the schedule applies to
exclude: [ string ] # which branches to exclude from the schedule
always: boolean # whether to always run the pipeline or only if there have been source code changes since the last successful scheduled run. The default is false.
The Schedule trigger does not support setting the target value for Parameters.
On the other hand, when you set the runtime parameters, the option or drop-down list to select the value can only be displayed when the pipeline is run manually.
I can fully understand your requirements.
You could add your request for this feature on our UserVoice site (https://developercommunity.visualstudio.com/content/idea/post.html?space=21 ), which is our main forum for product suggestions:
Related
I have a DevOps variable group with a variable like that: VARIABLE=['a', 'b', 'c'].
Then in Azure pipeline, there is a parameter like that:
parameters:
- name: parameter_test
displayName: 'Test paramter'
type: string
default: a
values:
- a
- b
- c
I want to use the variable instead of the hardcoded list, since it's present in multiple pipelines. Tried this, but docs say I can't use expressions in parameters section:
parameters:
- name: parameter_test
displayName: 'Test paramter'
type: string
default: a
values:
- ${{ each group in variables.VARIABLE }}:
- ${{ group }}
Have you ever tried things like that or have any idea how to parametrize it?
Thanks for any help!
According to this document Variable groups for Azure Pipelines - Azure Pipelines | Microsoft Docs, to reference a variable group, use macro syntax or a runtime expression, therefore the parameter cannot be defined with the value of variable from a variable group.
Instead of defining the parameter with the value of the variable in a variable group, you may consider using a core YAML to transfer the parameter/variable value into a YAML Template. Kindly refer to the below sample YAML pipeline.
# Core YAML
trigger:
- none
pool:
vmImage: ubuntu-latest
variables:
- name: Variable_core
value: b
parameters:
- name: Parameter_core
default: c
values:
- a
- b
- c
steps:
- template: Parameters.yml
parameters:
parameter_test_Azure_Repos_1: ${{ variables.Variable_core }}
parameter_test_Azure_Repos_2: ${{ parameters.Parameter_core }}
# Parameters.yml from Azure Repos
parameters:
- name: parameter_test_Azure_Repos_1
displayName: 'Test Parameter 1 from Azure Repos'
type: string
default: a
- name: parameter_test_Azure_Repos_2
displayName: 'Test Parameter 2 from Azure Repos'
type: string
default: a
steps:
- script: |
echo ${{ parameters.parameter_test_Azure_Repos_1 }}
echo ${{ parameters.parameter_test_Azure_Repos_2 }}
All non yaml files is not recommended as this is not as code, very difficult to check & audit & versionning, so as to variable group, release pipeline etc.
Azure pipeline has indeed some limitations, we can reuse the variables but not the parameters.
If I was you, even multiple pipelines use the same parameter, I will still "hard code" this directly in the pipelines just like what you wrote:
parameters:
- name: parameter_test
displayName: 'Test paramter'
type: string
default: a
values:
- a
- b
- c
Can azure devops pipeline yaml drive picklist values from a file so to avoid hardcoded values in yaml ? Even better allow an api call to dynamically populate a list
Naturally we can roll our own synthesis of pipeline yaml code short of that would be nice if it allowed an include snippets file ability ... one means would be to run a pipeline to generate a pipeline to get run
azure pipeline bad technique today requires a hardcode
parameters:
- name: some_parm
type: string
default: fav_value
values:
- val01
- val02
- val03
what is needed ... populate list dynamically or at a minimum from a file
parameters:
- name: some_parm
type: string
default: fav_value
values:
${{ some_file_or_api_lookup }}
Possibly this yaml preprocessor could work https://github.com/birchb1024/goyamp ... dunno yet
UPDATE 20220906 so far no solution found suggestions welcome
I am wanting opinion on the following. Have a javascript qna bot that I have in Azure DevOps. I have an azure pipeline created that deploys to an Azure environment. This works well. However, this is a common use bot that can be used in multiple scenarios. Write Once, Use Many. So I want to variabl-ize the process for multiple environments (DEV vs PROD) and instances (PROD1, PROD2, PROD3...)
1st Case: Within the project, there is a .env file with name-value pairs stored. I need to have distinct values for multiple environments and instances. One option could be to have a distinct file per environment+instance. So
.env.DEV, .env.PROD1, .env.PROD2, .env.PROD3, etc.
And then as part of the build process that zips the files, rename only one of the .env files by dropping the suffix based on the case. Can delete the other .env files prior to zipping. Is this a good way to do it OR is there a more standardized process that I should use?
2nd Case: As part of the deployment, I want to variabl-ize the azure-pipeline.yml file so that the target webapp, resource group, subscription, etc are dynamic (different for DEV, PROD1, PROD2, ...). I can create multiple yaml files and link it to separate pipelines. Is this the way? Or am I creating one pipeline and somehow toggling these values for 'n' different cases?
I can hack something. But I wanted to make sure I was using the right approach before starting.
Thanks in advance,
Jake.
1st Case:
Is this a good way to do it OR is there a more standardized process that I should use?
I suggest you can use replace token task to achieve your needs which could be more convenient. Here is my sample:
1.*.env file:
name1:#{value1}#
name2:#{value2}#
name3:#{value3}#
2.Create variables and set values when running the pipeline:
3.Replace token task:
- task: replacetokens#3
inputs:
targetFiles: '**/*.env'
encoding: 'auto'
writeBOM: true
actionOnMissing: 'warn'
keepToken: true
tokenPrefix: '#{'
tokenSuffix: '}#'
useLegacyPattern: false
enableTelemetry: false
4.Result of *.env file:
name1:a
name2:b
name3:c
2nd Case:
I can create multiple yaml files and link it to separate pipelines. Is
this the way? Or am I creating one pipeline and somehow toggling these
values for 'n' different cases?
I suggest you can use parameters and select values when running pipelines. For example:
parameters:
- name: subscription
type: string
default: test1
values:
- test1
- test2
- name: WebAppName
type: string
default: test1
values:
- test1
- test2
- name: ResourceGroupName
type: string
default: test1
values:
- test1
- test2
steps:
- task: AzureAppServiceManage#0
inputs:
azureSubscription: ${{ parameters.subscription }}
Action: 'Stop Azure App Service'
WebAppName: ${{ parameters.WebAppName }}
SpecifySlotOrASE: true
ResourceGroupName: ${{ parameters.ResourceGroupName }}
You can choose the resource group name and subscription name when running pipelins:
I have a pipeline (in YAML) which upgrades an infrastructure(I have 2 stages each containing a series of jobs)
I now want to upgrade multiple infrastructures simultaneously i.e. pass a list of identifiers which represents deployments to the pipeline and then let it upgrade each.
What is the best practice here for organising the pipeline? It feels like I need to generate a set of parallel jobs using a loop?
As I understand it any job failure will result in a total failure which could leave us in a very confused state.
If you purchased parallel jobs for your organization. You can use Template to generate multiple jobs according the identities parameter using expression ${{each id in parameters.identities}}.
So you can move the job which upgrades the infrastructures into a template and define your yaml pipeline as below. See below example:
Template file: upgrade-infrastructure.yml
parameters:
id: 1
jobs:
- job: upgradeinfra${{parameters.id}}
steps:
- powershell: echo "upgrade-infra-${{parameters.id}}"
azure-pipelines.yml:
#define the identities as a object to hold a array object of ids
parameters:
- name: identities
type: object
default:
ids:
- 1
- 2
trigger: none
stages:
- stage: Upgrage
pool:
vmImage: windows-latest
jobs:
- job: A
steps:
- powershell: echo "job A"
#loop through the ids array object and the each id to the template paramter to generate multiple jobs for each id.
#indentation is very important, bad indentation may cause pipeline compile error.
- ${{ each id in parameters.identities.ids }}:
- template: upgrade-infrastructure.yml
parameters:
id: ${{id}}
After you set up your yaml pipeline as above, you can enter the identities in the parameter when executing the pipeline:
Then you will see the multiple jobs are generated and run in parallel:
To make your deployments run in parallel, all you need to do is to set the dependencies. (The dependency on the previous step is automatically set). Here is an example of a stage that only depends on the build before all stage will run in parallel:
stage:
-stages : DeployTo${{ parameters.environment }}
dependsOn: ["Build"] //The stage that build the code is called "Build"
The Result looks like this:
Without the dependsOn property your pipeline stages will run sequentially and looks like this:
stages:
-stage : DeployTo${{ parameters.environment }}
Context
Azure Pipelines supports different kinds of CI triggers (docs).
Example A:
trigger:
branches:
include:
- master
The pipeline will run if new commits are detected in the master branch.
Example B:
trigger:
branches:
include:
- refs/tags/v*
The pipeline will run if new tags beginning with v are detected.
I want my pipeline to run when both of the above conditions are true: branch and tag.
Example C (not working as desired):
trigger:
branches:
include:
- master
- refs/tags/v*
Combining the two triggers seems to act like an or-condition, not an and-condition.
Question
How can I trigger a pipeline on the master branch only when there are new v* tags?
In my opinion it is not possible. Following documentation:
If you specify tags in combination with branch filters that include file paths, the trigger will fire if the branch filter is satisfied and either the tag or the path filter is satisfied.
Unfortunatelly adding condition will not help here because we don't have sufficient information. For instance for tag trigger we have this
BUILD_SOURCEBRANCH=refs/tags/release-07
BUILD_SOURCEBRANCHNAME=release-07
and for branch trigger this:
BUILD_SOURCEBRANCH=refs/heads/master
BUILD_SOURCEBRANCHNAME=master
And even if you try to check manually branch name with git branch you will get:
* (HEAD detached at 154ce86)
For me this is a good candidate for a feature request at developer community.
I found a way of doing this by using templates and parameters.
Example
First, create a common.yaml with parameterized tasks and scripts:
parameters:
- name: myMessage
type: string
default: 'Hello from template!'
- script: |
echo ${{ parameters.myMessage }}
This yaml can contain any number of parameters, shared tasks and scripts.
Then, create a master.yaml for the master branch:
trigger:
branches:
include:
- master
extends:
template: common.yml
parameters:
myMessage: 'Hello from master!' # Overrides default parameter value
Create another tags.yaml to run for each new version tag:
trigger:
branches:
include:
- refs/tags/v*
extends:
template: common.yml
parameters:
myMessage: 'Hello from tags!' # Overrides default parameter value
Lastly, create two Pipelines within Azure DevOps web interface. Connect one of the Pipelines to master.yaml and the other to tags.yaml.
With this setup, each trigger is independent with minimal yaml duplication.