I can pass the variable successfully from upstream to downstream pipeline. But I can't get the dynamic variable value for ref: $branch line. I'm getting the following error in the downstream pipeline: Project main/project reference does not exist! I think that somehow I can't give the value to ref section.
Upstream pipeline:
---
stages:
- test
test:
stage: test
variables:
branch: "dev"
trigger:
project: child/test
strategy: depend
branch: master
allow_failure: false
rules:
- if: $CI_COMMIT_BRANCH == 'master' || $CI_COMMIT_BRANCH == 'dev'
when: never
Child pipeline:
---
stages:
- hellotest
hellotest_1:
stage: hellotest
trigger:
include:
- project: 'main/project'
file: 'samplejob/example.gitlab-ci.yml'
ref: $branch
Related
I have the following GitLab pipeline. For brevity I have removed some code. In deploy-dev stage I am creating environment dev. Then in the next stage test-postdeploy I am using $CI_ENVIRONMENT_NAME to set the base_url variable value. However, in test-postdeploy stage the $CI_ENVIRONMENT_NAME always empty
stages:
deploy:dev
deploy:qa
deploy:prod
test:postdeploy
deploy-dev:
image: node
stage: deploy:dev
environment:
name: dev
script:
- . deploy/scripts/deploy-dev.sh
rules:
- if: $CI_COMMIT_REF_NAME == "dev"
deploy-qa:
image: node
stage: deploy:qa
environment:
name: qa
script:
- . deploy/scripts/deploy-qa.sh
rules:
- if: $CI_COMMIT_REF_NAME == "qa"
deploy-prod:
image: node
stage: deploy:prod
environment:
name: prod
script:
- . deploy/scripts/deploy-prod.sh
rules:
- if: $CI_COMMIT_REF_NAME == "release"
test-postdeploy:
tags:
- xlarge
image: node
stage: test:postdeploy
variables:
base_url: ""
script:
- echo "$CI_ENVIRONMENT_NAME" // this is empty
- script/postdeploy-test.sh
rules:
- if: $CI_ENVIRONMENT_NAME == "dev"
variables:
base_url: "https://dev.example.com"
- if: $CI_ENVIRONMENT_NAME == "qa"
variables:
base_url: "https://qa.example.com"
- if: $CI_ENVIRONMENT_NAME == "prod"
variables:
base_url: "https://prod.example.com"
UPDATE 1
I can probably pass environment name to another stage using https://docs.gitlab.com/ee/ci/variables/#pass-an-environment-variable-to-another-job
However, this does not make sense. Environment Name is most basic value that any ci/cd pipeline would need to know. The CI_ENVIRONMENT_NAME
variable should be available in all stages once its set without doing some extra work.
I have a pipeline like below:
include:
- project: "some-project"
ref: 0.5.0
file: reusable-jobs.yml
variables:
VAR1: ""
stages:
- stage-1
- stage-2
job1:
extends: .reusable-job-1
stage: stage-1
variables:
SOME_VAR: "$VAR1"
job2:
extends: .reusable-job-2
stage: stage-2
variables:
SOME_VAR: "$VAR1"
I want to be able to save this an imported into other projects. with my reusable jobs I create one like this
.some-reusable-job:
image:
name: alpine
script:
- echo "hello"
and then I extended it when I want to use it
ex:
reuse_the_job:
extends: .some-reusable-job
stage: some-stage
but I cannot figure out how to do the same thing with the entire pipeline including the stages
I want to be able to call the pipeline like so:
reuse_the_pipeline:
extends: .my-reusable-pipeline
variables:
VAR1: "hello"
and have the pipeline be created with both stage_1 and stage_2
I tried to create a definition like this in reusable-flow-file.yml
.reusable-flow
include:
- project: "some-project"
ref: 0.5.0
file: reusable-jobs.yml
variables:
VAR1: ""
stages:
- stage-1
- stage-2
job1:
extends: .reusable-job-1
stage: stage-1
variables:
SOME_VAR: "$VAR1"
job2:
extends: .reusable-job-2
stage: stage-2
variables:
SOME_VAR: "$VAR1"
and then use it like this (.gitlab-ci.yml)
include:
- local: "reusable-flow-file.yml"
dev_na:
extends: .reusable-flow
variables:
VAR1: "hello"
but when I try to run it and get lab I get this error
config should implement a script: or a trigger: keyword
not really sure what that try
It's because you're extending the ".reusable-flow" on a job-level. And what you're actually getting is (the following) which is invalid, because the job dev_na has no script property.
dev_na:
include:
- project: "some-project"
ref: 0.5.0
file: reusable-jobs.yml
variables:
VAR1: ""
stages:
- stage-1
- stage-2
job1:
extends: .reusable-job-1
stage: stage-1
variables:
SOME_VAR: "$VAR1"
job2:
extends: .reusable-job-2
stage: stage-2
variables:
SOME_VAR: "$VAR1"
variables:
VAR1: "hello"
You should just define the jobs (and other stuff) on the root level of your template. And play around a bit in the Project > CI/CD > editor where you can get a life preview of the templated configuration.
I have below pipeline.
After creating merge request, created Detached merge request pipeline with failed status (app1 - no stages/jobs).
In scope of below pipeline need to run pipeline when merge request is created and after merging changes main. Flow described in here Gitlab CI Child pipeline
Below pipeline does not work.
workflow:
rules:
- if: '$CI_PIPELINE_SOURCE == "schedule"'
- if: '$CI_PIPELINE_SOURCE == "web"'
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
- if: '$CI_PIPELINE_SOURCE == "push"'
stages:
- child-pipelines
app1:
stage: child-pipelines
variables:
COMPONENT NAME: 'app1'
trigger:
include:
- local: .ci/.gitlab-ci.yml
strategy: depend
rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
changes:
- test1/**/*
variables:
DEPLOY_RELEASE: '11111'
- if : '$CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_BRANCH == "main"'
changes:
- test1/**/*
variables:
DEPLOY_RELEASE: '222222'
app2:
stage: child-pipelines
variables:
COMPONENT NAME: 'app1'
trigger:
include:
- local: .ci/.gitlab-ci.yml
strategy: depend
rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
changes:
- test2/**/*
variables:
DEPLOY_RELEASE: '11111'
- if : '$CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_BRANCH == "main"'
changes:
- test2/**/*
variables:
DEPLOY_RELEASE: '222222'
If you want to run a pipeline when merge request was created and after merge to main branch, take a look this example:
workflow:
rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
- if: $CI_COMMIT_BRANCH == "main" && $CI_PIPELINE_SOURCE == "push"
job1:
script:
- echo "This job runs in merge request and also after merge to main branch"
According to Gitlab documentation.
If you want to run a rule for the entire pipeline:
workflow:
rules:
- if: $CI_PIPELINE_SOURCE == 'merge_request_event'
job1:
script:
- echo "This job runs in merge request pipelines"
job2:
script:
- echo "This job also runs in merge request pipelines"
If you want to run a rule in certain job:
job1:
script:
- echo "This job runs in merge request pipelines"
only:
- merge_requests
Problem
When the pipeline underneath is triggered, the Dev stage is not being run after Build and push.
The pipeline is triggered with a PR with develop as the target branch.
pipeline.yaml
trigger:
branches:
include:
- master
- develop
pr:
branches:
include:
- develop
stages:
# Frontend tests: Triggered by opening new PRs to develop or updating pr to develop.
- stage: FrontEndTests
displayName: "Frontend tests"
condition: eq( variables['Build.Reason'], 'PullRequest') # Trigger stage only for PullRequests
jobs:
- template: /templates/pipelines/npm-unit-tests.yml#templates
parameters:
triggerType: ${{ variables.triggerType }}
# Common build triggered by push to master or develop
- stage: BuildPush
displayName: "Build and push"
condition: ne(variables['Build.Reason'], 'PullRequest') # Don't perform stage if PR triggered pipeline
variables:
envName: "common"
jobs:
- template: /templates/pipelines/dockerbuild-dashboard-client.yml#templates
parameters:
displayName: "Build docker image"
deploymentName: "docker_build_push"
dependsOn: ""
# Dev deploy stage
- stage: dev
displayName: "Dev"
dependsOn: BuildPush
condition: and(succeeded(), ne(variables['Build.SourceBranch'], 'refs/heads/master'))
variables:
envName: "dev"
jobs:
- template: /templates/pipelines/webapprelease-dashboard-dev-client.yml#templates
parameters:
dependsOn: ""
deploymentName: "publish_container_to_webapp"
# Test deploy stage
- stage: test
displayName: "Test"
dependsOn: BuildPush
condition: and(succeeded(), ne(variables['Build.SourceBranch'], 'refs/heads/develop'))
jobs:
- template: /templates/pipelines/webapprelease-dashboard-test-client.yml#templates
parameters:
dependsOn: ""
deploymentName: "publish_container_to_webapp"
# Prod deploy stage
- stage: prod
displayName: "Prod"
dependsOn: test
condition: and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/master'))
variables:
envName: "prod"
jobs:
- template: /templates/pipelines/webapprelease-dashboard-prod-client.yml#templates
parameters:
dependsOn: ""
deploymentName: "publish_container_to_webapp"
Question
Why is the Dev stage not run? It seems to me that the conditions for the dev stage are not met, but I cannot see why.
In your case succeded is evaluated as
With no arguments, evaluates to True only if all previous jobs in the dependency graph succeeded or partially succeeded.
And FrontEndTests was skipped thus while was evaluated as false.
Please change it to
condition: |
and(
or
(
in(dependencies.FrontEndTests.result, 'Succeeded', 'SucceededWithIssues', 'Skipped'),
in(dependencies.BuildPush.result, 'Succeeded', 'SucceededWithIssues', 'Skipped')
),
ne(variables['Build.SourceBranch'], 'refs/heads/master'))
I tested this on this case and it works as above
pool:
vmImage: 'ubuntu-latest'
stages:
- stage: build
displayName: Build
condition: eq( variables['Build.Reason'], 'PullRequest')
jobs:
- job: Build
steps:
- bash: echo "Build"
- stage: test
displayName: Test
condition: succeeded()
jobs:
- job: Test
steps:
- bash: echo "Test"
- stage: test2
displayName: Test2
condition: |
or(
in(dependencies.build.result, 'Succeeded', 'SucceededWithIssues', 'Skipped'),
in(dependencies.test.result, 'Succeeded', 'SucceededWithIssues', 'Skipped')
)
jobs:
- job: Test2
steps:
- bash: echo "Test2"
I'm using Azure Pipelines as a part of Azure DevOps.
I'm trying to define variables in my template file, because I need to use the same value multiple times.
This is my stage-template.yml:
parameters:
- name: param1
type: string
- name: param2
type: string
variables:
var1: path/${{ parameters.param2 }}/to-my-file.yml
stages:
- stage: Deploy${{ parameters.param2 }}
displayName: Deploy ${{ parameters.param1 }}
jobs:
...
When trying to use this pipeline, I get an error message:
/stage-template.yml (Line: 7, Col: 1): Unexpected value 'variables'
Why is this not working? What did I do wrong?
You can't have variables in a template that is included as a stage, job or step template (i.e. included below a stages, jobs or steps element in a pipeline). You can only use variables in a variable template.
The documentation sadly is not really clear about that.
Including a stage template
# pipeline-using-stage-template.yml
stages:
- stage: stage1
[...]
# stage template reference, no 'variables' element allowed in stage-template.yml
- template: stage-template.yml
Including a variable template
# pipeline-using-var-template.yml
variables:
# variable template reference, only variables allowed inside the template
- template: variables.yml
steps:
- script: echo A step.
If you are using a template to include variables in a pipeline, the included template can only be used to define variables.
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/templates?view=azure-devops#variable-reuse
you cant have parameters in the pipeline, only in the templateReferences:
name: string # build numbering format
resources:
pipelines: [ pipelineResource ]
containers: [ containerResource ]
repositories: [ repositoryResource ]
variables: # several syntaxes, see specific section
trigger: trigger
pr: pr
stages: [ stage | templateReference ]
if you want to use variables in templates you have to use proper syntax:
parameters:
- name: param1
type: string
- name: param2
type: string
stages:
- stage: Deploy${{ parameters.param2 }}
displayName: Deploy ${{ parameters.param1 }}
variables:
var1: path/${{ parameters.param2 }}/to-my-file.yml
jobs:
...
This works for me:
In your parent yaml:
stages:
- stage: stage1
displayName: 'stage from parent'
jobs:
- template: template1.yml
parameters:
param1: 'somevalueforparam1'
inside template1:
parameters:
param1: ''
param2: ''
jobs:
- job: job1
workspace:
clean: all
displayName: 'Install job'
pool:
name: 'yourpool'
variables:
- name: var1
value: 'value1'