I would like to decide if a build is a scheduled job from inside the build.
For example, on Travis CI my build scripts look for the $TRAVIS_EVENT_TYPE environment variable to see if its value is cron.
How should I do this on CircleCI?
Use a custom environment variable (envar). While you can't set an envar directly in a workflow, you can add one to a CircleCI Context, and then attach that context to the workflow. The envar name can be whatever you want and the value whatever you want.
One example would be to create a context called nightly when an envar in it that you can then check for in a job. Here's an example:
nightly-build:
triggers:
- schedule:
cron: "0 0 * * *"
filters:
branches:
only: master
jobs:
- build:
context: nightly
Related
I'm trying to use a variable yaml file where I store a variable that is the cron syntax for a build. I wish to use this variable for multiple build pipelines, and want to be able to change the time/day of the build without having to go into each pipeline and change each schedule within each yaml pipeline.
However, trying this current method hits an error:
variable.yml
variables:
- name: cronSyntax
value: "0 9 * * *"
azure-pipelines.yml
variables:
- template: variable.yml
schedules:
cron: ${{ cronSyntax }}
etc
I have also tried doing $(cronSyntax) but neither seem to work. Is it just a case that I cannot use variables within the schedule task in yaml? Any help greatly appreciated.
Thanks
Looks like you can't use pipeline variables when specifying schedules.
See the official documentation here : https://learn.microsoft.com/en-us/azure/devops/pipelines/process/scheduled-triggers?view=azure-devops&tabs=yaml#scheduled-triggers
I have a GitLab CI pipeline with a 'migration' job which I want to be added only if certain files changed between current commit and master branch, but in my current project I'm forced to use GitLab CI pipelines for push event which complicates things.
Docs on rules:changes clearly states that it will glitch and will not work properly without MR (my case of push event), so that's out of question.
Docs on rules:if states that it only works with env variables. But docs on passing CI/CD variables to another job clearly states that
These variables cannot be used as CI/CD variables to configure a
pipeline, but they can be used in job scripts.
So, now I'm stuck. I can just skip running the job in question overriding the script and checking for file changes, but what I want is not adding the job in question to pipeline in first place.
While you can't add a job alone to a pipeline based on the output of a script, you can add a child pipeline dynamically based on the script output. The method of using rules: with dynamic variables won't work because rules: are evaluated at the time the pipeline is created, as you found in the docs.
However, you can achieve the same effect using dynamic child-pipelines feature. The idea is you dynamically create the YAML for the desired pipeline in a job. That YAML created by your job will be used to create a child pipeline, which your pipeline can depend on.
Sadly, to add/remove a Gitlab job based on variables created from a previous job is not possible for a given pipeline
A way to achieve this is if your break your current pipeline to an upstream and downstream
The upstream will have 2 jobs
The first one will use your script to define a variable
This job will trigger the downstream, passing this variable
Upstream
check_val:
...
script:
... Script imposes the logic with the needed checks
... If true
- echo "MY_CONDITIONAL_VAR=true" >> var.env
... If false
- echo "MY_CONDITIONAL_VAR=false" >> var.env
artifacts:
reports:
dotenv: var.env
trigger_your_original_pipeline:
...
variables:
MY_CONDITIONAL_VAR: "$MY_CONDITIONAL_VAR"
trigger:
project: "project_namespance/project"
The downstream would be your original pipeline
Downstream
...
migration:
...
rules:
- if: '$MY_CONDITIONAL_VAR == "true"'
Now the MY_CONDITIONAL_VAR will be available at the start of the pipeline, so you can impose rules to add or not the migration job
I have a Pipeline that has a few stages: detect, test, build, deploy
The detect stage detects the type of application and the test and build stages have jobs that are included or excluded based on what is computed in detect. The detect stage writes it's value to a environment variable called BUILD_MODE.
I am using rules like so:
ng-build:
extends:
- '.ng/job/build'
stage: build
rules:
- if: $BUILD_MODE == "ANGULAR"
when: always
npm-build:
extends:
- '.npm/job/build'
stage: build
rules:
- if: $BUILD_MODE == "NPM"
when: always
The problem with this is that the BUILD_MODE variable is evaluated statically when the pipeline is created not after the detect stage runs so the above never works unless I set the variable explicitly in the top level YML file like so:
variables:
BUILD_MODE: "ANGULAR"
What is the best way to solve this problem? The summary of what I want to do is evaluate some condition, either set the stages dynamically or set the variable itself before the stages in the Pipleline are created so they will be created with the rules evaluated correctly.
You could take a look at dynamic child-pipelines. Maybe you could solve your problem by dynamically creating your npm/ng build jobs.
I am planning to schedule jobs on runners based on variable from pipeline web UI. I have two runners registered for a project and currently both are on same machine for now.Both runners are having the same tags so differentiation from tags is not possible too.
This is my yaml file
stages :
- common
- specific
test1:
stage: common
tags:
- runner
- linux
script:
- echo $CI_RUNNER_DESCRIPTION
- echo "This is run on all runners"
test2:
stage: specific
tags:
- runner
- linux
script: echo "This is run on runner 1"
only:
variables:
- $num == $CI_RUNNER_DESCRIPTION
test3:
stage: specific
tags:
- runner
- linux
script: echo "This is run on runner 2"
only:
variables:
- $num == $CI_RUNNER_DESCRIPTION
So variable on which the selection happens is "num". The description of the runners can be 1 or 2.
The default value for num is 0 and according to variable passed from pipeline UI, the jobs are to be selected and run.
But when I execute the test with num 1 or 2, only test1 gets executed which is common to all runners.
Is such a implementation possible or am I facing issue because the runners are on same machine ?
Using variables only is not the right approach to select runners. Right now you select one of the 2 runners and try not to execute job if runner description doesn't match. Even if this worked it wouldn't make much sense as you don't know if the job will be executed or not.
I suggest you add specific labels for runners and specify which job should run on which runner.
test2:
stage: specific
tags:
- runner_1
script: echo "This is run on runner 1"
test3:
stage: specific
tags:
- runner_2
script: echo "This is run on runner 2"
If you want to select runner for the job based on UI input, maybe you can add variable to tags section. To be honest I didn't test such solution and donno if it's gonna work. Yet still it would require to create specific tags for your runners.
tags:
- $tag_variable_selected_from_UI
You cannot do this currently with gitlab. As you noted in your comment on the accepted answer, this will be available when https://gitlab.com/gitlab-org/gitlab/-/issues/35742 is resolved.
That being said, while the "accepted" answer correctly states "only" is the incorrect way to do this, it suggests another way that also doesnt work.
Once dynamic tagging support is added into gitlab-ci, this will be possible. Currently, the solution is to provide unique jobs each with their own tags.
Depending on what exactly you are trying to accomplish, you can keep your code as DRY as possible for now by adding job templates in combination with extends to reduce duplication to a couple of lines per job.
I have a simple gitlab-yaml file that I thought would run a job when only scheduled. However, it is getting fire on a push event as well.
Can anyone please tell me the correct way in which to specify that a job is only run when scheduled.
This is my gitlab-yaml file
job:on-schedule:
only:
- schedules
- branches
script:
- /usr/local/bin/phpunit -c phpunit_config.xml
Thanks
According to the GitLab documentation, branches means "When a branch is pushed".
https://docs.gitlab.com/ce/ci/yaml/README.html#only-and-except-simplified
So including branches in your only: section causes the pipeline job to also run on pushes to any branch.
You can either remove the branches entry, or if you wanted to restrict to pushes for a specific branch you could extend the branch entry to include project and branch name (branches#<project>/<branch>).
My suggestion is to reduce your YML to:
job:on-schedule:
only:
- schedules
script:
- /usr/local/bin/phpunit -c phpunit_config.xml