I think I'm fundamentally missing something. I'm new to CI/CD and trying to set up my first pipeline ever with gitlab.
The project is a pre-existing PHP project.
I don't want to clean it up just yet, at the moment I've pushed the whole thing into a docker container and it's running fine talking to google cloud's mysql databases etc as it should locally and also on a remote google cloud testing VM.
The dream is to be able to push to the development branch, and then merge the dev banch into the test branch which then TRIGGERS automated tests (easy part), and also causes the remote test VM (hosted on google cloud), to PULL the newest changes, rebuild the image from the latest docker file (or pull the latest image from gitlab image register)... and then rebuild the container with the newest image.
I'm playing around with gitlab's runner but I'm not understanding what it's actually for, despite looking through almost all the online content for it.
Do I just install it in the google cloud VM, and then when I push to gitlab from my development machine.. the repo will 'signal' the runner (which is running on the VM, to execute a bunch of scripts (which might include git pull on the newest changes?).
Because I already pre-package my app into a container locally (and push the image to the image registry) do I need to use docker as my executor on the runner? or can i just use shell and shell the commands in?
What am I missing?
TLDR and extra:
Questions:
What is runner actually for,
where is it meant to be installed?
Does it care which directory it is run in?
If it doesn't care which directory it's run,
where does it execute it's script commands? At root?
If I am locally building my own images and uploading them to gitlab's registry,
Do I need to set my executor to docker? Shouldn't I just set it to shell, pull the image, and build it? (Assuming the runner is runing on the remote VM).
What is runner actually for?
You have your project along with a .gitlab-ci.yml file. .gitlab-ci.yml defines what stages your CI/CD pipeline has and what to do in each stage. This typically consists of a build,test,deploy stages. Within each stage you can define multiple job. For example in build stage you may have 3 jobs to build on debian, centos and windows (in GitLab glossary build:debian, build:centos, build:windows). A GitLab runner clones the project read the gitlab-ci.yaml file and do what he is instructed to do. So basically GitLab runner is a Golang process that executes some instructed tasks.
where is it meant to be installed?
You can install a runner in your desired environment listed here. https://docs.gitlab.com/runner/install/
or
you can use a shared runner that is already installed on GitLab's infrastructure.
Does it care which directory it is run in?
Yes. Every task executed by runner is relativly to CI_PROJECT_DIR defined in https://gitlab.com/help/ci/variables/README. But you can alter this behaviour.
where does it execute it's script commands? At root?
Do I need to set my executor to docker? Shouldn't I just set it to shell, pull the image, and build it?
A runner can have mutiple executors such as docker, shell, virtualbox etc but docker being the most common one. If you use docker as the executor you can pull any image from docker hub or your configured registry and you can do loads of stff with docker images. In a docker environment normally you run them as the root user.
https://docs.gitlab.com/runner/executors/README.html
See gitlab access logs , runner is constantly polling the server
Related
We are running a GitLab with CI and want to deploy our application onto a Windows server. This will be just a demo system so we are quite open how we run the app later.
Currently we build and test the application. Now we need to stop the old app at the server, bring the new build onto server and start the new app.
I want to use at best the tools from GitLab but this isn't a must and as you can assume I am not so used to GitLab Deploy stuff.
Docker would be also an option.
Do you have some ideas?
Thank you
Nico
If you aren’t precious about the actual deployment of your web app then the easiest thing to do would just be install a gitlab runner on your test windows host. You should use a shell runner, so it can operate as a “fake user”. You’ll also need to ensure the user the runner is installed as has the right creds to install your app.
Tag this runner so it will run for a particular job, then within your gitlab ci you can take a copy of your build as an artifact (see gitlab ci artifacts).
Artifacts are automatically copied between runners so long as they are within the same pipeline run so you will have accsess to your built web app and can then install it how you like.
We have a locally deployed instance of Gitlab, where at the time we cannot afford to have a GitLab Runner to run CI/CD pipelines.
Is it possible to use GitLab Pages without CI/CD? E.g. is it possible to manually prepare the HTML-content, put it in the public folder in the root and make it accessible?
I assume you don't want to use GitLab shared runner which costs you and CI minutes limits.
Public folder
Refer to this gitlab-ci.yaml for making HTML content available https://gitlab.com/pages/plain-html/-/blob/master/.gitlab-ci.yml
Running locally without registration
Here is the known limitations: https://docs.gitlab.com/runner/commands/#limitations-of-gitlab-runner-exec
Pages are directly managed by Gitlab so running locally won't affect the page's state.
Use your own runner and register with gitlab (Possible)
Install GitLab runner on your machine and register as a runner, instructions can be found at Settings->CI/CD->Runners->Specific runner Enable it for your project or group.
I use my personal machine gitlab-runner to execute CI jobs that aren't counted towards CI minutes.
We have two servers in our organisation.
1) server with gitlab
2) Build server
I would like to create an automate build happen in the second machine(Build server ) for the source code in the gitlab server.
How can I achieve this using gitlab ?
Thanks,
siva
If you are moving from an "pull" continuous integration system (e.g. using a kind of crontab that regularly checks if the source code on the versioning system has changed and start the configure/build/test/deploy stages if it has), then know that gitlab has a much better way of doing this.
gitlab approach is to configure a "pull" system: every time the code is updated (in any branch) on the git repository then the script defined in your .gitlab-ci.yml is read to see if continuous integration jobs have to be launched. jobs are send to your configured gitlab runners. gitlab runners are defined on your build server(s) and takes the job when they are coming.
Definition of what to do is also describes in the .gitlab-ci.yml.
Here is a list of documentation to start learning about gitlab CI:
the official documentation can be helpful
A general introduction to gitlab ci using docker can be found in this blog article (the first slides are great). If your build server or your intended build is on Linux, I would recommend using the "docker executor" (e.g. gitlab runners are executed inside a docker machine inside your build server). It is easy and quick to setup.
Hope this helps you starting...
What's the differences between the VSTS Build tasks of docker (with preview) and docker without preview?
It's said in the description that 'red' ones can be used with Docker or Azure Container Registry, is this an only difference?
Could they differentiate with docker/compose version or environment (e.g., one for windows, one for linux?)
Based on the source code of them, the difference between them is adding supported for Azure Container Registry, the Docker Registry Connection are the same.
You can setup a private build agent and add these tasks to a build definition and queue build with that build agent, and then these tasks will be downloaded to the _task folder (e.g. _work_task) and you can check it.
Where does the Job script runs? Does it run on the same system as of Runner?
How the runner runs the script, and where does build get saved?
It would be great, if someone could explain the whole flow of Gitlab CI.
Yes, your jobs' scripts run on the same system as the runner. But before I go deeper, we need to discuss terminology. My team has run into issues before because the term 'runner' is overloaded. Humans typically use 'runner' to mean two different things:
A server on which the gitlab-ci exe is located
A gitlab-ci runner
The former should be self-explanatory; when you want to create a gitlab-ci runner, the first thing you do is provision a VM and put the exe on it somewhere.
The latter takes some explanation. gitlab-ci runners are not like Jenkins slaves; they're not entire servers. Instead, gitlab-ci runners act like a combination of workspaces and Jenkins labels. In other words, gitlab-ci runners combine a server, a gitlab instance, an execution environment and a set of tags. It is entirely possible, and in fact normal, to have multiple gitlab-ci runners on the same server.
The job script is completely decoupled from either type of runner (by 'job script' I assume you mean code called from a .gitlab-ci.yml file). Anything you call from that .gitlab-ci.yml, and in fact the script elements within it, will be executed
By a runner matching the tags configured for the job running the script
On the server on which that runner is installed
In a shell or Docker or Vagrant container, depending on the runner
Finally, the build is saved on the on which the runner is installed. the location will depend on the folder into which you dropped the gitlab-ci executable. Otherwise, jobs are stored on the file system in a manner reminiscent of Jenkins workspaces.