I need include yml file with logic that if branch with same name exist in other project then include yml from this branch, else include yml from master branch.
Is there a way to make that?
I think about using dynamic variable, but I don't know how to set it before include and after access to triggered ref name
include:
- project: my_other_project
ref: $branch_name_that_equals_current_branch_or_master_branch
file: my.yml
Check which variable you are using. According to the docs you can only use certain variables like predefined variables or those set manually when triggering the pipeline. Unfortunately you cannot use variables defined in the variables block.
So, your example will work if you use one of the variables mentioned here.
include:
- project: my_other_project
ref: $CI_COMMIT_REF_NAME
file: my.yml
For instance.
See
Use variables with include
Where variables can be used
In your yaml file
variables:
REF: master
stages:
- build
build:
stage: build
only:
refs:
- $REF
script:
- echo "Building branch $REF"
Related
I'd like to parameterize the environment URL of one of my GitLab CI job in a project which belongs to a subgroup.
If for example I have:
CI_PROJECT_PATH = mygroup/mysubgroup/myproject
CI_PROJECT_NAMESPACE = mygroup/mysubgroup
CI_PROJECT_NAME = myproject
I'd like to have the URL to be something like https://mygroup.gitlab.com/-/mysubgroup/myproject/-/jobs/12345/artifacts/public/index.html.
But I cannot find a way to do this since there is no predefined variable for the "sub-namespace" (here mysubgroup) and there is no variable substitution in environment.url as far as I can see.
I tried in my gitlab-ci.yml things like that:
build:
stage: build
image: bash:latest
script:
- export # print the available variables
environment:
name: test
url: ${CI_SERVER_PROTOCOL}://${CI_PROJECT_ROOT_NAMESPACE}.${CI_PAGES_DOMAIN}/-/${CI_PROJECT_PATH#${CI_PROJECT_ROOT_NAMESPACE}/}/-/jobs/$CI_JOB_ID/artifacts/public/index.html
but the result is https://mygroup.gitlab.com/-/${CI_PROJECT_PATH#myproject/}/-/jobs/12345/artifacts/public/index.html.
References:
https://stackoverflow.com/a/58402821/1064669
https://gitlab.com/gitlab-org/gitlab/-/issues/350902
job:environment:url is directly processed by gitlab, which does not support this kind of parameter expansion.
You'll need to use bash or a similar shell for this to work as intended:
build:
stage: build
image: bash:latest
script:
- echo "DEPLOY_URL=${CI_SERVER_PROTOCOL}://${CI_PROJECT_ROOT_NAMESPACE}.${CI_PAGES_DOMAIN}/-/${CI_PROJECT_PATH#${CI_PROJECT_ROOT_NAMESPACE}/}/-/jobs/$CI_JOB_ID/artifacts/public/index.html" > deploy.env
- cat deploy.env
artifacts:
reports:
dotenv: deploy.env
environment:
name: test
url: $DEPLOY_URL
Based on the example given here: https://docs.gitlab.com/ee/ci/environments/#example-of-setting-dynamic-environment-urls
I think you need to escape the inner curly braces like this: (${CI_PROJECT_ROOT_NAMESPACE} should become $`{CI_PROJECT_ROOT_NAMESPACE`})
So the whole thing becomes:
url: ${CI_SERVER_PROTOCOL}://${CI_PROJECT_ROOT_NAMESPACE}.${CI_PAGES_DOMAIN}/-/${CI_PROJECT_PATH#$`{CI_PROJECT_ROOT_NAMESPACE`}/}/-/jobs/$CI_JOB_ID/artifacts/public/index.html
I need to pass a file path to a trigger job where the file path is found within a specified json file in a separate job. Something along the lines of this...
stages:
- run_downstream_pipeline
variables:
- FILE_NAME: default_file.json
.get_path:
stage: run_downstream_pipeline
needs: []
only:
- schedules
- triggers
- web
script:
- apt-get install jq
- FILE_PATH=$(jq '.file_path' $FILE_NAME)
run_pipeline:
extends: .get_path
variables:
PATH: $FILE_PATH
trigger:
project: my/project
branch: staging
strategy: depend
I can't seem to find any workaround to do this, as using extends won't work since Gitlab wont allow for a script section in a trigger job.
I thought about trying to use the Gitlab API trigger method, but I want the status of the downstream pipeline to actually show up in the pipeline UI and I want the upstream pipeline to depend on the status of the downstream pipeline, which from my understanding is not possible when triggering via the API.
Any advice would be appreciated. Thanks!
You can use artifacts:reports:dotenv for setting variables dynamically for subsequent jobs.
stages:
- one
- two
my_job:
stage: "one"
script:
- FILE_PATH=$(jq '.file_path' $FILE_NAME) # In script, get the environment URL.
- echo "FILE_PATH=${FILE_PATH}" >> variables.env # Add the value to a dotenv file.
artifacts:
reports:
dotenv: "variables.env"
example:
stage: two
script: "echo $FILE_PATH"
another_job:
stage: two
trigger:
project: my/project
branch: staging
strategy: depend
Variables in the dotenv file will automatically be present for jobs in subsequent stages (or that declare needs: for the job).
You can also pull artifacts into child pipelines, in general.
But be warned you probably don't want to override the PATH variable, since that's a special variable used to help you find builtin binaries.
I have a .gitlab-ci.yml file that says:
include:
- project: 'my-proj/my-gitlab-ci'
ref: master
file: '/pipeline/gitlab-ci.yml'
Because of some "Inconvenience" I would like to override some specific stage that is defined on the above mentioned gitlab-ci.yml' file injected on my top level .gitlab-ci.ymlfile. Theplan` stage I am interested in has the following thing:
plan-dummy:
stage: plan
script:
- terraform plan -lock=false -var-file=vars/vars.tfvars
What I want to do is override the above on the main .gitlab-ci.yml file such that only the script is executed as an override:
plan-dummy:
stage: plan
script:
- terraform refresh # This is the line I want to add as an additional step before next
- terraform plan -lock=false -var-file=vars/dev.tfvars
How do I achieve that without fiddling with the injected file? Yes, I know that alternative is to do dirty copy-paste from child file, but I don't want to do that.
Regards,
simply reuse the same job name and add the configuration you need:
plan-dummy:
before_script:
- terraform refresh
You can do the terraform refresh in a before_script part which will be executed before you script:
plan-dummy:
extends: plan-dummy
before_script:
- terraform refresh
I am using below two gitlab repository
Parent Gitlab repo - Application code, for example - Angular application
Child Gitlab repo - For Gitlab Pipeline, has only gitlab-ci.yml file which contain script to run pipeline
I am calling pipeline/child-project gitlab-ci.yml file form parent using below steps
Parent Gitlab repo - gitlab-ci.yml file
include:
- project: 'my-group/child-project'
ref: master
file: '/templates/.gitlab-ci-template.yml'
Child-project - gitlab-ci.yml file
stages:
- test
- build
before_script:
- export PARENT_PROJECT_NAME = ?
- export PARENT_PROJECT_PIPELINE_ID = ?
- export PARENT_PROJECT_BRANCH_NAME = ?
job 1:
stage: test
script:
- echo "Runnig test for project ${PARENT_PROJECT_NAME}"
- node_modules/.bin/ng test
release_job:
stage: build
script: node_modules/.bin/ng build --prod
artifacts:
name: "project-$CI_COMMIT_REF_NAME"
paths:
- dist/
only:
- tags
How can I get the parent-repo details like parent-project name, pipeline-id & branch name in child-project which is running the pipeline?
One way is to define the variables in parent-project and use in child project, but is there any other way where we can directly access the parent-project detail in the child-project?
In your example, include is not the same as trigger. Include just merges all the files together into one giant pipeline so you should be able to access any variables you want from the included files, as long as the variable's scope is correct.
If you are actually looking to pass details to a child pipeline from a parent pipeline you could add a job that exports the variables and details you want to the dotenv, then have the child pipeline access that dotenv. This would allow the code to be dynamic inside of hard coding in the variables and directly passing them to the child pipelines
export-parent-details:
script:
- echo "PARENT_PROJECT_NAME=?" >> build.env
- echo "PARENT_PROJECT_PIPELINE_ID=?" >> build.env
- echo "PARENT_PROJECT_BRANCH_NAME=?" >> build.env
artifacts:
reports:
dotenv: build.env
trigger-child:
stage: docker_hub
trigger:
include:
- project: 'my-group/child-project'
ref: master
file: '/templates/.gitlab-ci-template.yml'
# use this variable in child pipeline to download artifacts from parent pipeline
variables:
PARENT_PIPELINE_ID: $CI_PIPELINE_ID
Then inside the child jobs, you should be able to access the parent artifacts from the parent
child-job:
needs:
- pipeline: $PARENT_PIPELINE_ID
job: export-parent-details
script:
- echo $PARENT_PROJECT_NAME
See
https://docs.gitlab.com/ee/ci/yaml/README.html#artifact-downloads-to-child-pipelines
https://docs.gitlab.com/ee/ci/multi_project_pipelines.html#pass-cicd-variables-to-a-downstream-pipeline-by-using-variable-inheritance
Another option could be to make an API call to the get the parent project's details since the runners have a read-only token under $CI_JOB_TOKEN, this method is dependent on repo access privileges and the details you want
curl -H "JOB_TOKEN: $CI_JOB_TOKEN" "https://gitlab.com/api/v4/{whatever the api call is}"
Since you’re including the child project’s configuration instead of triggering it, the two pipeline definition files are merged and become one before the pipeline starts, so there’s be no practical difference between this method and having the content of the child project’s definition in the parent’s.
Because of this, all of the predefined variables will be based on the parent project if the pipeline runs there. For example, variables like $CI_COMMIT_REF_NAME, $CI_PROJECT_NAME will point to the parent project and the parent project's branches.
Currently in our project the gitlab-ci.yml includes a common template yml with default stages and job definitions. I want to include a stage and respective job specific to this project, which needs to run in between the stages defined in template yml. Below are the sample yml files which represent the scenario I am working on.
.mvn-template.yml
stages:
- build
- static-analysis
- unit-test
- package
- integration-test
compile:
stage: build
tags:
- docker
artifacts:
paths:
- target/
expire_in: 14 days
script:
- mvn $MAVEN_CLI_OPTS compile
interruptible: true
//and job definitions for - static-analysis
// - unit-test
// - package
// - integration-test
gitlab-ci.yml
include:
- project: 'xxx/common-pipeline'
ref: x.x.x
file: '/.mvn-template.yml'
...
Now I want include a new stage specific this project, say contract-testing, in between
package
integration-test
So the contract-testing should run after package and before integration-test.
As this is specific to this project I cannot include in common template. Thus needs to be included in gitlab-ci.yml.
Could not figure out how to do this. I earlier worked with GitHub and just recently started on project with GitLab.
Can I include the stage in common template but don't define respective job there. And include the Job definition in gitlab-ci.yml? Not sure if this is the correct way. Probably there could be better options to handle such scenario.
Please help.
In my opinion, you can do it in 2 different ways :
Add the stage contract-testing in your template file and define the specific job in the .gitlab-ci.yml of your project. The configuration will be valid : gitlab will concatenate and expand includes/anchors to build one single gitlab-ci at the end. Note : you can define a stage even if there is no job using this stage (a sort of an empty stage), it will not generate an error.
If you don't want to have this "empty stage" included in all your projects referring the include, you can move the stages outside of you template and put them in the .gitlab-ci.yml of your project. Like for solution 1), you need to define the job in .gitlab-ci.yml of your project (not in the template).