My Gitlab CI jobs fail because of RAM limitations.
Page https://docs.gitlab.com/ee/user/gitlab_com/index.html says:
All your CI/CD jobs run on n1-standard-1 instances with 3.75GB of RAM, CoreOS and the latest Docker Engine installed.
Below it says:
The gitlab-shared-runners-manager-X.gitlab.com fleet of runners are dedicated for GitLab projects as well as community forks of them. They use a slightly larger machine type (n1-standard-2) and have a bigger SSD disk size. They don’t run untagged jobs and unlike the general fleet of shared runners, the instances are re-used up to 40 times.
So, how do I enable these n1-standard-2 runners (which have 7.5 GB RAM)? I've read the docs over and over but can't seem to find any instructions.
Disclaimer: I did not check if you can use them with a project and if they picked up for your gitlab CI/CD - but that is the way, of how you can check for available Runners and their tags, and how to use them. The terminology GitLab projects as well as community forks of them reads like that this is only for Official GitLab projects and their forks - and not for random projects on GitLab.
You can check all the available runners in your projects CI/CD Settings under Runners, and you will see a list of runners there like:
As you can see there are Runners tagged with gitlab-org. Base on the description you can not run them, without using a tag. Hence that you need to adapt your .gitlab-ci.yml file with those appropriate tags.
EG:
job:
tags:
- gitlab-org
see GitLab documentation for tags
Related
I am facing an out of memory error in one of my CI/CD pipelines, so I would like to customise the configuration of my GitLab's shared runners, for example by using a config.toml file. I would also like to avoid self-hosting a GitLab Runner instance, if possible.
Is there a way of doing that ?
As far as I know, there is no way of changing the config.
However, according to this doc, I can pick from 3 machine sizes up to 16GB RAM, and add the appropriate tag at the job level in my gitlab-ci.yaml.
Note that this will impact the CI/CD minutes cost factor.
For GitLab Premium and Ultimate (os, not free), you do have GitLab 15.4 (September 2022), which comes with:
More powerful Linux machine types for GitLab SaaS runners
When you run jobs on GitLab SaaS Linux runners, you now have access to more powerful machine types: medium and large. With these two machine types, you have more choices for your GitLab SaaS CI/CD jobs. And with 100% job isolation on an ephemeral virtual machine, and security and autoscaling fully managed by GitLab, you can confidently run your critical CI/CD jobs on GitLab SaaS.
See Documentation and Issue.
I'm already using gitlab CI on smaller projects, but now I'm looking into using gitlab as CI for a larger project.
How can I pass build artifacts (bunch of binary files etc) between two gitlab-runners running on two different physical machines?
Context:
I have a large repository, which produces a lot of artifacts during the build. Obviously this takes time, so I'd like to build on a beefy multi-core machine. If the build passes, I want to test in parallel across many other (smaller) machines. These test-machines are hooked up to many different kinds of equipment. Equipment that I don't want to bother the beefy machine with.
I understand artifacts: and dependencies: should address this, but that uses a local cache as far as I can tell.
The build artefacts weigh in at ~4GB so somehow that data must be transferred.
Can gitlab help with this natively, or do I need a pattern of build+push followed by a fetch+test? (To say, artifactory CEPH NFS etc)
I imagine my needs aren't unique so something must already exist for this.
You are on the right path: artifacts is what you are looking for. Runners do not store the artifacts they build, but they upload them to the GitLab instance.
Now, where GitLab stores them is a different topic, and if you manage your GitLab installation, you can take a look at the administration documentation: https://docs.gitlab.com/ee/administration/job_artifacts.html
You can also retrieve artifacts through APIs, if you have any special need, but artifacts and dependencies should be more than enough for your use case: https://docs.gitlab.com/ee/api/job_artifacts.html#get-job-artifacts
I am new to gitlab CI and I am fascinated with it. I managed already to get the pipelines working even using docker containers, so I am familiar with the flow for setting jobs and artifacts. I just wish now to understand how this works. My questions are about the following:
Runners
Where is actually everything happening? I mean, which computer is running my builds and executables? I understand that Gitlab has its own shared runners that are available to the users, does this mean that if a shared runner grabs my jobs, is it going to run wherever those runners are hosted? If I register my own runner in my laptop, and use that specific runner, my builds and binaries will be run in my computer?
Artifacts
In order to run/test code, we need the binaries, which from the build stage they are grabbed as artifacts. For the build part if I use cmake, for example, in the script part of the CI.yml file I create a build directory and call cmake .. and so on. Once my job is succesful, if I want the binary i have to go in gitlab and retrieve it myself. So my question is, where is everything saved? I notice that the runner, withing my project, creates something like refs/pipeline/, but where is this actually? how could I get those files and new directories in my laptop
Working space
Pretty much, where is everything happening? the runners, the execution, the artifacts?
Thanks for your time
Everything that happens in each job/step in a pipeline happens on the runner host itself, and depends on the executor you're using (shell, docker, etc.), or on the Gitlab server directly.
If you're using gitlab.com, they have a number of shared runners that the Gitlab team maintains and you can use for your project(s), but as they are shared with everyone on gitlab.com, it can be some time before your jobs are run. However, no matter if you self host or use gitlab.com, you can create your own runners specific for your project(s).
If you're using the shell executor, while the job is running you could see the files on the filesystem somewhere, but they are cleaned up after that job finishes. It's not really intended for you to access the filesystem while the job is running. That's what the job script is for.
If you're using the docker executor, the gitlab-runner service will start a docker instance from the image you specify in .gitlab-ci.yml (or use the default that is configurable). Then the job is run inside that docker instance, and it's deleted immediately after the job finishes.
You can add your own runners anywhere -- AWS, spare machine lying around, even your laptop, and jobs would be picked up by any of them. You can also turn off shared runners and force it to be run on one of your runners if needed.
In cases where you need an artifact after a build/preparatory step, it's created on the runner as part of the job as above, but then the runner automatically uploads the artifact to the gitlab server (or another service that implements the S3 protocol like AWS S3 or Minio). Unless you're using S3/minio, it will only be accessible through the gitlab UI interface, or through the API. In the UI however, it will show up on any related MR's, and also the Pipeline page, so it's fairly accessible.
How can I determine what is the minimum hardware requirement for install the gitlab-runner on a host?
I want a specific runner which will run tests, build docker images, and deploy the apps.
Gitlab documentation gives some advices and put focus on memory :
Depending on how you decide to configure GitLab Runner and what tools you use to exercise your application in the CI environment, GitLab Runner can consume significant amount of available memory.
After that, it's really depending the type of application you want to test/build/deploy. Test and build job will be the largest resource consumers. You can also set limits on cpu/memory for docker runner.
In high performance computing is crucial to have a code tested against many different architectures/compilers: from a laptop to a supercomputer.
Assuming that we have
N testing machines/workers (each one running gitlab-ci-runner);
M tests,
what shall be the correct layout of .gitlab-ci.yml to ensure that each of the N machines runs all the tests?
Looks to me that adding just more workers ends up in a round-robin like assignment of the jobs.
Thanks for your help.
You could use tags in your .gitlab-ci.yml and on the runners to distribute your tests to all mashines you want. Gitlab CI is very open to those use cases. I assume that you don't use docker for testing.
To accomplish your goal, do following steps:
Install your gitlab-ci-multi-runners on your N mashines BUT if the runner ask you for tags, tag the mashine with a specifc name e.g. "MashineWithManyCPUsWhatever". Please use the gitlab-ci-multi-runner register command to do so. You can alternatively change the tags in Gitlab on the administration view after registration.
The type of the runners should be "shared" not "specific".
When you installed and tagged every runner on your mashines, tag you jobs.
Example:
Every job will now be excecuted on the specific mashine with the specific tag.
For more Information see Gitlab CI Documentation