I'm trying to use a variable yaml file where I store a variable that is the cron syntax for a build. I wish to use this variable for multiple build pipelines, and want to be able to change the time/day of the build without having to go into each pipeline and change each schedule within each yaml pipeline.
However, trying this current method hits an error:
variable.yml
variables:
- name: cronSyntax
value: "0 9 * * *"
azure-pipelines.yml
variables:
- template: variable.yml
schedules:
cron: ${{ cronSyntax }}
etc
I have also tried doing $(cronSyntax) but neither seem to work. Is it just a case that I cannot use variables within the schedule task in yaml? Any help greatly appreciated.
Thanks
Looks like you can't use pipeline variables when specifying schedules.
See the official documentation here : https://learn.microsoft.com/en-us/azure/devops/pipelines/process/scheduled-triggers?view=azure-devops&tabs=yaml#scheduled-triggers
Related
I've found the following block:
stages:
- stage: Print_Params
jobs:
- job: Print_Params
steps:
- ${{ each parameter in parameters }}:
- script: echo ${{ parameter.Key }} ${{ parameter.Value }}
But it invokes CmdLine once for each specified parameter. I'd really like to have a single screen I can look at to review all the parameters that a pipeline was invoked with. Is this built in, and there's a place I can already review it, or is there a way I can invoke the loop within a script to print all of the parameters in a single execution? I've tried a number of different syntaxes and nothing I've tried so far is working.
You can view runtime parameters, queue time variables and job preparation parameters in Azure Pipelines UI:
I know that you can reuse blocks of code in a before script using yaml anchors:
.something_before: &something_before
- echo 'something before'
before_script:
- *something_before
- echo "Another script step"
but this doesn't seem to work when the .something_before is declared in a shared .yml file via the include:file. It also does not seem that extends works for before_script. Does anyone know a way of reusing some steps in a before_script from a shared .yml file?
EDIT: My use case is that I have 2 gitlab projects with almost identical before_script steps. I don't want to have to change both projects whenever there's a change, so I have a third, separate Gitlab project that has a .yml template that I am including via include:file in both projects. I want to put all the common code in that shared template, and just have like two lines before_script for the git project that has the two extra steps.
Yaml anchors don't work with included files. You need to use the extends keyword. But what you want to achieve won't work with before_script as code in your template will be overwritten in the job which uses the template if there is a before_script as well.
Do you really need a before_script in your specific job or can you achieve the same with a normal script? If yes you can do something like this:
Template File:
.something_before:
before_script:
- echo 'something before'
- echo 'something more before'
Project Pipeline:
include:
- project: 'my-group/my-project'
file: '/something_before.yml'
stages:
- something
something:
stage: something
extends: .something_before
script:
- echo 'additional stuff to do'
And your before_script section will be merged into the something job and executed before the script part.
See if GitLab 13.6 (November 2020) does make it easier:
Include multiple CI/CD configuration files as a list
Previously, when adding multiple files to your CI/CD configuration using the include:file syntax, you had to specify the project and ref for each file. In this release, you now have the ability to specify the project, ref, and provide a list of files all at once. This prevents you from having to repeat yourself and makes your pipeline configuration less verbose.
See Documentation) and Issue.
And even, with GitLab 14.9 (March 2022):
Include the same CI/CD template multiple times
Previously, trying to have standard CI/CD templates that you reuse in many places was complicated because each template could only be included in a pipeline once.
We dropped this limitation in this release, so you can include the same configuration file as many times as you like.
This makes your CI/CD configuration more flexible as you can define identical includes in multiple nested configurations, and rest assured that there will be no conflicts or duplication.
See Documentation and Issue.
You can use extends without any problem, but you will need to overwrite the entire before_script block.
If you want to change just a piece of your before_script, use a shell script to do it
Set the if condition inside of your template
before_script
- |
if [ condition ]
then
commands here
fi
AFTER EDIT: You can use variables to achieve it
Project 1: VAR = command 1
Project 2: VAR = command 2
You can set the content of env var on the gitlab-ci.yml file or on the CI/CD settings in each project!
I'm trying to import variable groups depending on the current branch in azure build pipeline, but I get this error: 'Object reference not set to an instance of an object'.
I have simplified the case and I get this error when I have both of the lines (condition and import) in my .yaml file:
variables:
${{ if eq('bbb', 'master') }}:
- group: variables-shared
If I remove condition, everything works as expected. If I remove group importing, I get other errors related to undefined variable below (that is normal).
I am interested why I get this error
I also had this exact issue. The reasoning in the currently accepted answer is not correct, and you can in fact use conditional insertion in your azure-pipelines.yml file to conditionally include a particular variable group.
When the docs say that conditional insertion requires the use of template syntax, they're referring to template expression syntax, not the use of templates directly. Per the previous link, template expression syntax is the ${{ }} syntax for expanding expressions.
As gleaned from this example from the docs, The problem with the example in the question is actually a syntax error.
Incorrect:
variables:
${{ if eq('bbb', 'master') }}:
- group: variables-shared
Correct:
variables:
- ${{ if eq('bbb', 'master') }}:
- group: variables-shared
Note the leading - before the $ char on the second line. I've tested this in my own pipelines and it works, although it does freak out the yaml syntax checker in my IDE.
I actually discovered a hack yesterday for debugging this obnoxiously vague and wide-ranging error message.
For the build that fails with this message, if you click "Run new" and try to run the job manually by clicking "Run", it will typically give you a much more specific error message at that point.
For instance:
If I remove condition, everything works as expected.
I am interested why I get this error
Check the Expressions doc and you will find this: Conditionals only work when using template syntax.
That's why you can not use condition for your variable group.
The workaround is to use the template to store your variables rather than variable groups.
Please refer to Variable reuse:
# File: vars.yml
variables:
favoriteVeggie: 'brussels sprouts'
# File: azure-pipelines.yml
variables:
- template: vars.yml # Template reference
steps:
- script: echo My favorite vegetable is ${{ variables.favoriteVeggie }}.
I found two other potential causes to "An error occurred while loading the yaml build pipeline. Object reference not set to an instance of an object."
"stages:" was missing in stage template before the stage
Template file reference didn't exist (- template: 'template.yml')
I had also an object reference not set to an instance of an object with code 606802. The build pipeline had no errors at all.
The error was caused by a pre-validation build, where 1 parameter value had no default value.
After adding the default value, the PR validation build succeeded.
I have a lot of default parameters in my template. I want to categorize them.
# template.yml
parameters:
azure:
name: cargo_test # Default job name
displayName: Cargo test # Default displayName
condition: true # Job condition
strategy: # Default strategy to test on Windows, MacOs and Linux.
matrix:
Linux:
vmImage: ubuntu-16.04
MacOS:
vmImage: macOS-10.13
Windows:
vmImage: vs2017-win2016
name: job_name
default_parameter1: default1
default_parameter2: defualt2
# rest of code
- job:A
template: template.yml
parameters:
azure:
name: test_name
This cause parameter.azure contains only one field name. I want to overwrite parameters.azure.name not all parameters.azure struct. Is it possible in azure pipelines?
I want to overwrite azure.name not all azure struct.
It seems that you are worrying if you just overwrite one parameter in .yml file which is using other template.yml file, it will affect all azure struct, right?
If so, you don't need worry about this. As what you defined in template.yml file, it has lots of parameters. After you use it in other .yml file: name: test_name , it only overwrite the value of parameter name with no effect on other parameters, and also this overwrite only available on current job.
For example, if in your use-template.yml:
- job:A
template: template.yml
parameters:
azure:
name: test_name
- job:B
template: template.yml
parameters:
azure:
condition: failed()
The overwriting of name, will only affect this parameter(name) value in Job A. After Job A finished, the value of name will reback to cargo_test in Job B.
In one word, the configuration in template.yml is fixed, the used in other yml will have any affect to the template.yml. So, you don't need to worry about how to categorize parameters which we does not support it until now.
You can check this simple example in official doc: Job templates.If have any misunderstanding about your idea, just feel free to correct me.
Updated:
Since we can get the value with parameters.azure.name, the Azure Devops should support these parameters categorize. And also, after tested, I got the same result with you. If overwrite parameters.azure.name, the rest parameters which in the same level with parameters.azure.name are all empty. I think this should be a issue which need our Product Group to fix it.
I have raise this issue report on our official Developer Community: When I overwrite the template parameter, the value be empty. You can follow this ticket, so that you can get the notification once it has any updated.
In addition, it seems no other work around to achieve parameters categorize. Just patience for this issue fixed. Once the fixed script release, our engineer would inform it in that ticket.
Is it possible that I can create a variable via a batch file and pass it to a Gitlab CI variable?
The background is that I want to declare the link of an environment:
environment:
name: staging
url: https://staging.example.com
a part of the URL results dynamically from the current build date. How can I pass the variable declared in a batch file to the gitlab-ci "url" variable?
The Url would look like this in the batch file:
https://testme.com/Tool_%date:~-2%%date:~-7,2%%date:~-10,2%.zip
Outcome is:
https://testme.com/Tool_180410.zip
and that variable i want to write in the environment URL variable
I don't think creating a variable in a batch file and passing it to a GitLab CI variable is possible, but from what I've gathered from your scenario you could:
set the current build date before you call the batch script
pass in the variable to the batch script
use the variable for your URL generation
For example (running on a Windows runner):
$ set testDate=%date:~-2%%date:~-7,2%%date:~-10,2%
$ echo %testDate%
180410
# Use %testDate% wherever else you need it now for the rest of your build.
With regards to the Environment URL, I don't have any experience using it - but this open issue could be of interest.
I usually set any variable needed in Gitlab's .gitlab-ci.yml in another separate yml file. Let's assume that the name of this second file is "parameters.yml", just to give an example.
You can create the parameters.yml file from your batch and define all your variables there.
Example of what parameters.yml contains:
variables:
name: staging
url: https://staging.example.com
[other variables]
Then, all you have to do is to include this yml into the "main" one (.gitlab-ci.yml), for example, something like this:
stages:
- build
- test
- release
- deploy
include: parameters.yml
And that's it, now you will "see" all the variables defined in "parameters.yml"