I've been trying to get more familiar with GitLab's CI functionality and find the idea of splitting up a CI pipeline into multiple separate jobs interesting. This would allow me to maintain one project of "known jobs" and include them in other projects.
So far, I have something like this:
$ ls
jobA.yaml jobB.yaml jobC.yaml jobD.yaml
Those 4 are all identical (for now), and have the following:
job-name:
stage: my-stage # Might be needed to differentiate later on
tags: runner-tag # used to figure out where/how the job should be done: directly on a server, in a container, etc
script:
- echo "beep beep"
In the actual .gitlab-ci.yaml I want to use, I would then (I think) put something like this. In this case, I would use the jobs defined in the project for itself:
include:
project: '$CI_PROJECT_PATH'
file: "*.yaml"
stages:
- my-stage
That gives me back a linter error though. Perhaps I'm misreading the documentation, but I think that should be possible somehow....
This should be a comment, but can't put formatted code in there..
We use a main yml, which just include all the others. It is not wildcards like you have.
Have you tried changing "file" to "local"? with the leading "- "?
include:
- template: Code-Quality.gitlab-ci.yml
- local: '/.gitlab/py.yml'
- local: '/.gitlab/static.yml'
- local: '/.gitlab/lint.yml'
- local: '/.gitlab/docs.yml'
- local: '/.gitlab/publish.yml'
According to the docs, wildcard includes are only possible with local. Furthermore you need to move your jobA.yaml to a directory as otherwise you will include your .gitlab-ci.yml as well with a wildcard on the top level.
So the following works with JobA.yaml in config:
include:
- local: 'config/*.yaml'
stages:
- my-stage
Related
I have written a Git pipeline gitlab.yaml, where I am having both except and only rules.
I have a design/ folder which I am ignoring for most of the jobs and, except this folder, all my pipeline should get executed.
only:
refs:
- master
except:
changes:
- design/*
But: when I make changes to design folder file, and other files, then all the jobs are not getting executed.
Note: depending on your GitLab version (or with gitlab.com), you need to rewrite your exclusion rules using rules:
See "Transitioning your only/except syntax to rules".
The question "Gitlab CI how to ignore directory using rules syntax?".
However, as illustrated with gitlab-org/gitlab issue 301070, this new syntax does not solve your issue.
Currently it seems not to be possible to use rules:changes: to trigger a job when certain globs match, ignoring other sets of globs.
So for now, this is still being discussed/fixed.
You can try this glob pattern:
job:
rules:
- if: $CI_COMMIT_BRANCH == "master"
changes:
- '[!design]*/**'
I have a gitlab job that does not seem to update the repository before being run. Sometimes it leaves some files in their old states and run the script... Any idea ?
For instance when I have a
packagePython:
stage: package
script:
- .\scripts\PackagePython.ps1
tags:
- myServer
cache:
paths:
- .\python\cache\
only:
changes:
- python/**/*
I finally managed to understand what was happening :
I realised that the gitlab-runner did not use exactly the same path for each run on my server, and my script assumed that it did... So I ended up pointing on a build made on the wrong path.
I guess if you think that it is not updating the repository (like I did) make sure you are not referencing hardcoded path/package in your scripts that could refer to previous versions !
In my .gitlab-ci.yml i'm trying to deploy on merge request.
My pipeline works, script is executed , everything is ok and running , but i'm not able to read any predefined environment variables. My files looks like :
executeAutomationTests:
stage: check
only:
refs:
- merge_requests
script:
- echo $CI_MERGE_REQUEST_SOURCE_BRANCH_NAME
But $CI_MERGE_REQUEST_SOURCE_BRANCH_NAME is not resolved. I need to know source branch for the merge_request in order to pull the code and make a deployment. I have tried other variables like : $CI_COMMIT_REF_NAME, $CI_JOB_STAGE , but non of them are resolved.
My GitLab version : GitLab Community Edition 13.4.2
The problem was that in the pipeline job monitor the variable wasn't resolved and that's confused me. It was looking like :
So, for anyone who is fighting with such things, keep that in mind and also, keep in mind differences in syntax between powershell , bash and so on ..
I'm working with SVN but I would like to move on to Git, and more specifically to Gitlab.
I have the following structure:
MyStructure/
customer/
client1/
delivery.sh
MyFiletoSend.sh
client2/
delivery.sh
MyFiletoSend2.sh
Currently, the "delivery.sh" will send the modifications (rsync) of the file "MyFiletoSend.sh" to the server "client1".
Can I run the "delivery.sh" via Gitlab automatically after/before the git push only on the files modified in this push?
Example:
I have a modification to make to the file "MyFiletoSend.sh" from client1/
I make my change
commit and push
Gitlab is running "delivery.sh" on my "client1/" file.
The file "MyFiletoSend.sh" is sent to the server of "client1" without touching "client2".
Yes, it is possible
but first of all you need to understand how gitlab ci works. Read this article https://docs.gitlab.com/ee/ci/yaml/
You will create a step in your pipeline that will do what you want after you push the code (in master or in any other branch/mr)
and about the job? you have to create one, you can use this code to help you
https://gist.github.com/hnlq715/6c222ba0fd868bae7e4dfd3af61bf26e
Assuming your delivery.sh scripts have all the rsync logic required, GitLab has built-in logic to detect changes in files and execute bash commands in response. You can create a separate job for each client, which can run in parallel in the same stage. This approach is also auditable in that it will clearly show you which clients got updated and with which version of the file.
update-client-1:
stage: update-clients
only:
changes:
# Detect change only in MyFiletoSend.sh:
- customer/client1/MyFiletoSend.sh
# Detect any change in the customer folder:
- customer/client1/*
script:
- cd customer/client1
- delivery.sh
update-client-2:
stage: update-clients
only:
changes:
- customer/client2/*
script:
- cd customer/client2
- delivery.sh
# repeat for all remaining clients
For more information: https://docs.gitlab.com/ee/ci/yaml/#onlychangesexceptchanges
I have task to create simple GitLab CI file to run a job only if two files have been changed. In official GitLab docs I have not found nothing about this. I need to run a job only if file A changed and also file B changed.
In this case, you don't have to change by yourself, every time if you change/commit your File A and File B, GitLab CI will allow your files to automatically run as a job
Option A - Only/Except https://docs.gitlab.com/ee/ci/yaml/#onlyexcept-advanced
This is the correct way to do it
your-step:
script:
- your-script.sh
only:
changes:
- "file*.extension" # I don't know the pattern of your files
Option B - Using shell script
You can do anything with shell script (or any other language)
in this link https://gitlab.com/gitlab-org/gitlab-foss/-/issues/19813 you will see a lot of options to do it. Just try if something fits to your purpose
Option C - Using gitlab rules https://docs.gitlab.com/ee/ci/yaml/#rules
In the example below, the job will be triggered if any of the conditions match
your-step:
script:
- your-script.sh
rules:
- changes:
- fileA
- file B
- dir-with-both-files/*