I'm trying to "mutualize" my CI.
I have a lot of project that use basically the same gitlab-ci.yml.
I was thinking about creating a git submodule containing the gitlab-ci.yml that all my project are using, in which I could inject env variable in order to make it generic so I could re-use it in my futur projects.
My problem is that I don't see any documentation about this so I was wondering :
Is it possible ? And if yes, how can I tell gitlab to use the gitlab-ci.yml that is inside of this submodule ?
If it's a bad idea, do you have any idea of the correct way to do this ?
Thanks !
What you're looking for are Gitlab CI template files, that can be include'd in other .gitlab-ci.yml files.
You can check out the documentation for include here.
Example :
# .gitlab.ci.yml
include:
- 'https://gitlab.com/awesome-project/raw/main/.before-script-template.yml'
- '/templates/.after-script-template.yml'
This includes CI files with default variables, stages, workflows that can be overriden anywhere in your other CI files. This brings the genericity you need.
Related
I would like to receive some help on how I could create some environment variables to be used in README.md within a locally hosted Gitlab instance.
I need two variables to be replaced in README.md when someone accesses the web interface, variables that define the name of the repo and the name of the branch.
Any idea is welcome.
Thanks!
I don't think you can use these variables inside a GitLab readme. There is a feature request for this but it isn't implemented yet.
A way around this is to use the predefined variables that are present in GitLab. However, these variables are accessible to GitLab's CI pipelines, and not to any readme files. But perhaps you can find a solution in this answer. It suggests that you keep a placeholder in your readme file, then have a job run that switches out the readme's placeholder with the required value using the sed command. Since the job will have access to both the readme file and the variables, this should work.
I am a beginner to learn gitlab CI/CD and doing some projects related to it. I have a trouble when using OpenFaaS and build a large number of function.I want to use CI/CD to automatically build functions that have updates, remain functions are not, but I don't know how to get name of specific folder or directory changed in my source to build corresponding functions.
So I wonder if we have a way to get name of specific directory changed in Gitlab CI/CD? Or another tools CI/CD can do that?
only:changes is what you are looking for:
#.gitlab-ci.yml
my job:
script: echo "I am triggered only when changes to some/dir are applied"
only:
changes:
- some/dir/**/*
We have hundreds of similar projects in GitLab which have the same structure inside.
To build these projects we use a one common TeamCity build. We trigger and pass project GitLab URL along with other parameters to the build via API, so TeamCity build knows which exact project needs to be fetched/cloned. TeamCity VCS root accepts target URL via parameter.
The question is how to replace existing TeamCity build with a GitLab pipeline.
I see the general approach is to have CI/CD configuration file(.gitlab-ci.yml) directly in project. Since the structure of the projects the same this is not the option to duplicate the same CI/CD config file across all projects.
I'm wondering is it possible to create a common pipeline for several projects which can accept the target project URL via parameter ?
You can store the full CICD config in a repository and put in all your projects a simple .gitlab-ci.yml which includes the shared file.
With thus approach there is no redundant definition of the jobs.
Still, you can add specific other jobs to specific projects (in the regarding .gitlab-ci.yml files or define variables in a problem and use some jobs conditionally) - you can also include multiple other definition files, e.g. if you have multiple similar projects.
cf. https://docs.gitlab.com/ee/ci/yaml/#include
With latest GitLab (13.9) there are even more referencing methods possible: https://docs.gitlab.com/ee/ci/yaml/README.html#reference-tags
As #MrTux already pointed out, you can use includes.
You can either use it to include a whole CI file, or to include just certain steps. in Having Gitlab Projects calling the same gitlab-ci.yml stored in a central location - you can find detailed explanation with examples of both usages
I'm trying to set up a CI job that require dependent repositories to be placed along side the repository for which I'm enabling CI. By dependent, I mean that my main repo needs the code in the dependent repo but there is no build or test dependency between the two repos
I find a way to clone a dependent repository using this command in the job's script
git clone https://gitlab-ci-token:${CI_JOB_TOKEN}#gitlab.mycompany.com/path_to_my/dependent_repo.git
The problem is that the repo is fresh cloned every time the job runs which takes way too long as the dependent repo is quite large.
Is there a way to "fetch" a dependent repo as efficiently as GitLab CI fetches its own repo (against which CI will run), basically performing a pull instead of a clone?
Should I use cache?
If you have one main repo that depend on a few other repos, I would add these as submodules. This makes it easier to handle in GitLab, and your colleagues can easily find the correct versions they need to clone for these repos. If you have some specific need to not have these as submodules, then I understand!
In GitLab, there are a few different ways of handling this. The most simple one is to use the GIT_STRATEGYvariable:
https://docs.gitlab.com/ee/ci/yaml/#git-strategy
You can set it to fetch like this:
my_job:
variables:
GIT_STRATEGY: fetch
script:
- echo test
GitLab will then try to reuse an existing working directory instead of always cloning a new one.
I had a case myself when I used a flag for git clone called --reference:
https://git-scm.com/docs/git-clone#Documentation/git-clone.txt---reference-if-ableltrepositorygt
It has some very strange special cases that you have to think about. What the flag does is that git uses a local copy of a repository and copies objects from that one, instead of always copying from the network. This can greatly speed up cloning operations in some cases.
In addition to these suggestions, GitLab has a page with their suggestions to handle large repositories:
https://docs.gitlab.com/ee/ci/large_repositories/
I can see that GitLab supports various templates for things such as licenses, issue templates and such, but I can't seem to find a way to add a template for .gitlab-ci.yml files. Can this be done on Gitlab CE?
No, unless you can contribute (through merge request) to the gitlab-org/gitlab-ci-yml project, which is a collection of useful .gitlab-ci.yml templates.
Then, it is possible the API allows you to query that new template:
See CE Get GitLab CI YML templates
For one template:
curl https://gitlab.example.com/api/v4/templates/gitlab_ci_ymls/Ruby