GitLab CI, monorepo and feature branch - gitlab

I have a monorepo in GitLab with a Feature Branch approach.
What I'm trying to achieve is to launch the part of the pipeline associated to the directory containing the altered files. So my .gitlab-ci.yml looks like :
job1:
stage: build
script:
- ...
only:
changes:
- myparentdir/dir1/*
job2:
stage: build
script:
- ...
only:
changes:
- myparentdir/dir2/*
Create a new branch from develop
Commit myparentdir/dir2/test.txt on this branch
The pipeline launch every build jobs !
It seems like GitLab is considering every files as altered when working with a new feature branch.
Do you know any workaround?

Gitlab ci always treats changes for new branches as true. The reason is that they can't decide what to compare against.
what this means is that for the first pipeline of a new branch, everything will be built.
See the feature request for more details.
But, there is a fairly new feature called Pipelines for merge requests - that runs a stage on merge requests. Here is the feature request that implements changes with merge_requests. It's merged, but I'm not sure if it's already released. (the milestone is 11.9 - the next release)
In the meantime you can implement it yourself - You can add a stage that compares the changes (git diff) and decides whether to run the next stage:
.store_diff_from_main: &store_diff_from_main |
git diff --name-only origin/master...HEAD > "${DIFF_FILE}"
git diff --name-only HEAD~1 >> "${DIFF_FILE}"
.skip_stage: &skip_stage_condition |
echo Checking for changes in ${STAGE_PATHS}, changed files
# https://coderwall.com/p/gecfwa/git-diff-vs
cat .diff-from-master
# also cover merge squash cases
if ! (cat ${DIFF_FILE} | grep -E "${STAGE_PATHS}"); then
echo "Skipping stage ..."
exit 0
fi

Related

Gitlab auto pull

I have repository in GitLab. and I have Test and Dev branches in this repository.
In Gitlab pipeline, I schedule a job to auto run Test branch pipeline every 24 hours.
in Test .gitlab-ci.yml I have
deploy:
stage: deploy
script:
- git pull --ff-only origin Dev
only:
- Test
After merging Dev in Test, this part of code was removed. And next time Test branch pipeline could not pull from Dev branch, when pipeline run.
How can I pull code from Dev branch to Test Branch without losing the code - git pull --ff-only origin Dev ?
Or maybe it is possible to have tow .gitlab-ci.yml on branch? (if yes, how GitLab should know which one of them be diploid first? )
Or maybe it is possible to have two .gitlab-ci.yml on branch
You would generally create your .gitlab-ci.yml in the main/master branch.
Which means that .gitlab-ci.yml would not be impacted by merges between Test and Dev. Which is what you need.

Already up to date gitlab pipeline stackoverflow

i trying gitlab pipeline. now i make some changes & code pushed in master branch
pipeline showing already update to date but i have changes in code
I try to pull in three phase but still same issue
.gitlab-ci.yml
before_script:
- echo "Before script"
building:
stage: build
script:
- git pull origin master
testing:
stage: test
script:
- git pull origin master
deploying:
stage: deploy
script:
- git pull origin master
If the gitlab-ci workflow starts by cloning your repository, no amount of git pull will change the fact you already have the full history, and, at the time of the workflow, this is "already up to date".
In other words, a git pull would not be needed in your gitlab-ci.yml file.
If your pipeline is running on the same repo that you changed, there is no need to use git pull. Although, if your pipeline triggers (on repo A) another pipeline on another repository (repo B), to access files in repo A, you have to pull repo A in repo B pipeline.

GitLab CI/CD yml - How to add into pipeline job if new branch created. And exclude for further pushes?

In our git process we have hothix process. The process is to create a hotfix brunch from master and make changes and then merge to master back.
The issue that I'm faсing now is how to add job to pipelne if branch just created and exclude from pipeline for any further push?
When branch is created we need to set up custom environment. Create a sandbox, push code and push some test data. This is need only when branch is created. In case of changes and pushes, environment is available and no need in building it again. Just build code and push a release tag
I'm trying something like this:
job-name:
stage: build
script:
- myScript
rules:
- if: '$CI_COMMIT_BRANCH =~ /^hotfix.*/'
changes:
- force-app/**/*
when: never
- if: '$CI_COMMIT_BRANCH =~ /^hotfix.*/'
allow_failure: false
when: manual
And for sure it's not working as changes: not part of if policy:
Rules are evaluated in order until a match is found. If a match is found, the attributes are checked to see if the job should be added to the pipeline.

Gitlab - how to allow multi project pipelines to create a branch in trigger repo

I have a repo A(upstream) and a repo B(downstream). After some jobs complete in repo A; I want to trigger some jobs in repo B. I was able to achieve that.
trigger_repo_B:
stage: trigger_repo_B
trigger:
project: test/repo_B
What I havent been able to figure out is - how do I go about triggering repo B jobs for a non-existent branch in repo B. For example I can trigger jobs in repo B for a specific branch C if C exists but if C does not exist the pipeline is in a pending state. I want to be able to create a branch in B and then run the jobs in B if the branch C does not exist.
trigger_repo_B:
stage: trigger_repo_B
trigger:
project: test/repo_B
branch: C
Any ideas? The only way I could think of it working is to do a before_script where I clone the repo and create a branch before triggering the pipeline in B
Instead of listing the branch name in the trigger config, instead pass a variable where the value is the branch you want to create. Then in your downstream pipeline, you could add a stage and job to run before everything else that checks to see if the variable exists. If not, just exit 0 and let the pipeline continue as usual. But if it is set, create the branch, then continue.
That could look something like this:
#repo_A .gitlab-ci.yml
stages:
- trigger_repo_B
trigger:
stage: trigger_repo_B
variables:
BRANCH_TO_CREATE: branch_name
trigger:
- project: test/repo_B
#repo_B .gitlab-ci.yml
stages:
- prep_work
- build
Create Branch:
stage: prep_work
script:
- if [ -z ${BRANCH_TO_CREATE+x} ]; then git checkout -b $BRANCH_TO_CREATE; fi
# see https://stackoverflow.com/questions/3601515/how-to-check-if-a-variable-is-set-in-bash to explain the conditional above
Build:
stage: build
script:
- ...

Disable pipeline for every commit in Gitlab and only run it on open merge request

The CI pipeline runs on every commit in my Gitlab repository at work. Is there way to disable that and only run the CI pipeline on an open merge request to the master branch?
There is currently no configuration option to do that. Here are some things that can be used to "disable" a pipeline build.
Adding [ci skip] inside the commit message will not trigger a pipeline on push.
Using except and only options on all jobs inside the pipeline. To avoid duplication in this case, you can use Anchors.
Update: GitLab 11.7
When pushing to GitLab you can skip triggering a pipeline by passing ci.skip option to the push command: git push -o ci.skip
Update in 2020 because solution with only and except are candidates for deprecation : https://docs.gitlab.com/ee/ci/yaml/#onlyexcept-basic
Still in .gitlab-ci.yml you should use now rules : See https://docs.gitlab.com/ee/ci/yaml/#rules
I simply set this rule on my build job and the job is "blocked" and wait a manual trigger from the UI.
rules:
- when: manual
Note that we can create more advanced rules with conditions to trigger for exemple if we see a git tag.
https://docs.gitlab.com/ee/ci/yaml/#exclude-jobs-with-rules-from-certain-pipelines
Yes, but only if you set it on each job within the pipeline.
https://docs.gitlab.com/ee/ci/yaml/#onlyexcept-basic
job:
script: echo 'test'
only:
- merge_requests
That job will only run on commits that are part of a merge request, rather than every push of a commit to a branch. If you set every job to this setting then the pipeline will never run.
You can just add [ci skip] to the commit message and builds will be skipped
https://gitlab.com/gitlab-org/gitlab-ce/issues/14499
You could add something like this in the beginning of your .gitlab-ci.yaml
workflow:
rules:
- if: "$CI_COMMIT_BRANCH"
when: never
- if: '$CI_PIPELINE_SOURCE == "merge_request_event" && $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == "master"'
For more info look at Merge request pipelines
No. (Not yet !)
You can follow the expected feature development here.
https://gitlab.com/gitlab-org/gitlab-ce/issues/23902
However, you can automatically trigger pipelines using "maofr"'s code
https://gitlab.com/gitlab-org/gitlab-ce/issues/23902#note_88958643
For merge we can use this guide #onlyrefs--exceptrefs
merge_requests
For pipelines created when a merge request is created or updated. Enables merge request pipelines, merged results pipelines, and merge trains.
and for generally triggering pipeline we can use only: and variables: together:
script:
- . diff.sh $SMS_MESSAGE
only:
variables:
- $ACTION == "diff"
then triggering it is easy using curl:
curl -X POST \
-F token=$TOKEN \ # your token
-F ref=$REF_NAME \ # branch name
-F "variables[ACTION]=diff" \ # variable
https://gitlab.com/api/v4/projects/0/trigger/pipeline # your project endpoint
This part variables[ACTION]=diff will assign diff into ACTION and it will apply to .gitlab-ci.yml and only: section.
job:
stage: build
script:
- echo "Do your build here"
except:
- pushes
Use except pushes to stop create a new pipeline on every pushed commits. For reference check this
Update 2022 :
According to the documention of Gitlab on: https://docs.gitlab.com/ee/ci/yaml/#when
you can use when: manual to run the job only when triggered manually.
Example:
deploy_job:
stage: deploy
script:
- make deploy
when: manual
You don't need also rules for that.

Resources