When i start a build in gitlab ci (docker executer) its creating container with name "runner-0fa22293-project-174-concurrent-0-postgres"
In this what does "0fa22293-project-174-concurrent-0" represent ?
What is the naming convention used here?
I believe the format is as follows:
runner-<token>-project-<project ID>-concurrent-<concurrent job #>
<token> is the first 8 characters of the Runner Token for the CI runner that is running the build.
<project ID> is the ID of the project. The is the project ID you'd use during API access too.
<concurrent job #> is the number of this job, for this build. GitLab CI lets you run several jobs in a build concurrently, so if you had 3 of 5 jobs running concurrently you would see ...-concurrent-0, ...-concurrent-1, ...-concurrent-2
Related
Hi people of the internet.
Basically I am unable to run even the simplest job and I keep getting the same error no matter what I put in the .gitlab-ci.yml file. See example below:
Here is the .gitlab-ci.yml file:
stages:
- test
job1:
stage: test
tags:
- testing
script:
- echo "Hello world!"
Here is the output ("?" corresponds to intentionally blacked out information):
Running with gitlab-runner 14.10.0 (c6bb62f6)
on runner_test ????????
Preparing the "shell" executor
00:00
Using Shell executor...
Preparing environment
00:00
Running on LAPTOP-????????...
Getting source from Git repository
00:01
WriteError:
Line |
219 | $HOST="[MASKED]"
| ~~~~~~~~~~~~~~~~~~~~~~
| Cannot overwrite variable Host because it is read-only or constant.
ERROR: Job failed: exit status 1
I know that $HOST is a reserved variable in powershell but I don't see the link between the error and the code. It may have something to do with the configuration of the runner on Windows. Has anyone encountered this error on Gitlab before? Or any suggestions on how to debug?
Here are the steps that I took to install the runner on Gitlab for Windows (see https://docs.gitlab.com/runner/install/windows.html):
Create a folder somewhere in the system: C:\GitLab-Runner.
Download the binary for 64-bit and put it into the folder (see https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-windows-amd64.exe).
Run prompt as an administrator
Run the following command:
cd C:\GitLab-Runner
gitlab-runner.exe register
Enter your GitLab instance URL (see Gitlab > Settings > CI/CD > Runners > Specific runners)
Enter the token to register the runner (see Gitlab > Settings > CI/CD > Runners > Specific runners)
Enter a description for the runner: runner_test for instance
Enter the tags associated with the runner, separated by commas: testing, windows for instance
Provide the runner executor: shell
Install GitLab Runner as a service and start it
cd C:\GitLab-Runner
gitlab-runner.exe install
gitlab-runner.exe start
I also had to install the latest version of pwsh in Windows (see gitlab-runner: prepare environment failed to start process pwsh in windows):
Run prompt as an administrator
Install the newer pwsh.exe:
winget install Microsoft.PowerShell
Restart the runner
cd C:\GitLab-Runner
gitlab-runner.exe restart
This issue was due to my choice of shell for some reason. A Gitlab runner can choose a shell among the following: bash, sh, powershell, pwsh, and cmd (the last one being deprecated now).
As I stated above I had been using pwsh. So, I went after the config.toml file inside of the C:\GitLab-Runner directory to manually make the change from pwsh to powershell.
...
[[runners]]
name = "runner_test"
executor = "shell"
shell = "powershell"
...
I then restarted the runner and got the job to complete properly:
cd C:\GitLab-Runner
gitlab-runner restart
I still get the error (more like a warning now) but it does not prevent the job from finishing anymore. If anyone has a better answer with a proper explanation I would gladly accept it as the answer to this question.
Note that pwsh to powershell are both powershell scripts (see https://docs.gitlab.com/runner/shells/index.html):
powershell Fully Supported PowerShell script. All commands are executed in PowerShell Desktop context. In GitLab Runner 12.0-13.12, this is the default when registering a new runner.
pwsh Fully Supported PowerShell script. All commands are executed in PowerShell Core context. In GitLab Runner 14.0 and later, this is the default when registering a new runner.
Our Business Central extension is located on GitLab, and I've been trying to get the CI/CD feature to work. What I'd like to do is to use BcContainerHelper to set up a BC container and run a Test Runner of my choice before letting the pipeline succeed. Unfortunately, I've found very little documentation on this subject, and what I've found was either about BC pipelines in Azure, or pipelines for other projects in GitLab.
When creating the docker container, is it possible to set it up with BcContainerHelper instead so I have BC set up right away? Also, I tried to execute a .ps1 file, but I got the following error message:
Checking out fb0f1471 as 2-implementing-testapp...
Skipping Git submodules setup
Executing "step_script" stage of the job script
Using docker image sha256:13fd310aa3adfd5db7b986cc64b5b6816bea774cf51de468d917e6ef038b418f for ruby:2.5 with digest ruby#sha256:d273723056dda84bda81454eb42743c6c29fdf2c2d4d42bddf8e3dca8bb99aa4 ...
$ ./scripts/create-container.ps1
/bin/bash: line 120: ./scripts/create-container.ps1: Permission denied
Cleaning up file based variables
ERROR: Job failed: exit code 1
Is it even possible to run a pipeline for Business Central on GitLab or do I have to use Azure?
I don't know much about GitLab (we use Auzre DevOps with ALOps in our organization) but I can provide my two cents about BcContainerHelper.
BcContainerHelper has a command that will run a full build pipeline (creates the container, compiles the app, runs the tests). It is called Run-AlPipeline.
An example of running it could be:
Run-AlPipeline `
-pipelineName "My Pipeline" `
-licenseFile "c:\temp\license.flf" `
-baseFolder "...path to your project..." `
-appFolders #("app") `
-testFolders #("test") `
-installTestFramework `
-enablePerTenantExtensionCop `
-enableUICop
You can read about all the parameters available in this blog post.
I have one gitlab runner configured for a single project. The issue that I am seeing is that the runner will not wait until the prior job finished, and instead does a checkout in the same directory as the prior job and stomps over everything. I have one job already running, and then another develop commits and thus another job is started. Why can't I configure the pipeline not to run so that it doesn't corrupt the already running workspace?
Here is the log from both of the jobs (only difference is the timestamp)
[0K] Running with gitlab-runner 12.6.0 (ac8e767a)
[0K] on gitlab.xxxx.com rz8RmGp4
[0K] section_start:1578357551:prepare_executor
[0K] Using Docker executor with image my-image-build ...
[0K] Using locally found image version due to if-not-present pull policy
[0K] Using docker image sha256:xxxxxxxxxx for my-image-build ...
[0;msection_end:1578357553:prepare_executor
[0Ksection_start:1578357553:prepare_script
[0K] Running on runner-rz8RmGp4-project-23-concurrent-0 via gitlab.xxxx.com...
section_end:1578357554:prepare_script
[0K] section_start:1578357554:get_sources
[0K[32;1mFetching changes with git depth set to 50...[0;m
Initialized empty Git repository in /builds/my-project/.git/
<proceeds to checkout and stomp over the already running runner>
Main issue I see is that they both checkout to the same directory of Initialized empty Git repository in /builds/my-project/.git/ which causes the problem.
You can use resource_group to keep jobs from running in parallel.
e.g.
Job 1:
stage: My Stage
resource_group: stage-wedge
...
Job 2:
stage: My Stage
resource_group: stage-wedge
...
In the above example Job 2 will run after Job 1 is finished.
Jobs of same stage are executed in parallel.
if you need it to be sequential, you may add a stage for each of those jobs.
see the docs
https://docs.gitlab.com/ee/ci/yaml/#stages
In case of multiple pipelines running you may want to configure your gitlab-runner options: limit / concurrent
https://docs.gitlab.com/runner/configuration/advanced-configuration.html
I'm running a Laravel api on my server, and I wanted to use Gitlab-runner for CD. The first two runs were good, but then I started to see this problem listen_address not defined, session endpoints disabled builds=0
I'm running a linux server on a web shared hosting, so I can access a terminal and get some priviliges but I can't do some sudo stuff like installing a service. That's why I've been running gitlab-runner in user-mode
Error info
Configuration loaded builds=0
listen_address not defined, metrics & debug endpoints disabled builds=0
[session_server].listen_address not defined, session endpoints disabled builds=0
.gitlab-runner/config.toml
concurrent = 1
check_interval = 0
[session_server]
session_timeout = 1800
[[runners]]
name = "CD API REST Sistema SIGO"
url = "https://gitlab.com/"
token = "blablabla"
executor = "shell"
listen_address="my.server.ip.address:8043"
[runners.custom_build_dir]
[runners.cache]
[runners.cache.s3]
[runners.cache.gcs]
I have literally wasted 2 days on this subject. I have followed the below steps to get the runners configured and execute jobs successfully.
I am using Mac OS X 10.13 and Git Lab 12. However, people with other OS also can check this out.
I have stopped the runners and uninstalled them. Now deleted all references and files to gitlab runner, including the gitlab executable also.
I got to know GitLab Runner executable paths from https://docs.gitlab.com/runner/configuration/advanced-configuration.html
I have installed them again using the gitlab official documentation.
Then the runners shows online in the gitlab portal. However, the jobs are not getting executed. It shows simply stuck. It tried to get information from logs using
gitlab-runner -debug run
Then I got to know that listen_address not defined. After a long try I got to know that simply enabling Run Untagged jobs did the trick. The jobs started and completed successfully. Still the I see the listen_address not defined from debug. So that misled me.
Though it seems that last one task has solved my problem, but doing all the tasks in a batch did the trick.
Conversely, an alternative to Avinash's solution is to include the tags you create when you register the runner in the gitlab-ci.yml file
stages:
- testing
testing:
stage: testing
script:
- echo 'Hello world'
tags:
- my-tags
How to configure GitLab CI to store the SSTATE_DIR and the DL_DIR between several jobs? Currently, bitbake rebuilds the complete project every time, which is very time consuming. So i would like to use the sstage again. I tried caching, but building time increases effectively, due to the big zip/unzip overhead.
I don't even need a shared sstate between several projects, just a method to store the output between jobs.
I'm using Gitlab 11.2.3 with a shell executor as runner.
Thanks a lot!
In version 11.10, GIT_CLEAN_FLAGS was added, which could be used to achieve what you want to do with the shell executor.
For completeness: when using the docker executor, this can be achieved by using docker volumes, which are persistent across builds.
If you're only using one runner for this, you could potentially use GIT_STRATEGY: none, which would re-use the project workspace for the following job; relevant documentation. However, this wouldn't be extremely accurate in case you have multiple jobs running which requires the runner, as it could dilute the repository, if jobs are started from across different pipelines.
Another way, if you're still using one runner, is you could just copy the directories out and back into the job you require.
Otherwise, you may potentially be out of luck, and have to wait for the sticky runners issue.
You can reuse a shared-state cache between jobs simply as follows:
Specify the path to the sstate-cache directory in the .yml file of your
gitlab-ci pipeline. An example fragment from one of mine:
myrepo.yml
stages:
...
...
variables:
...
TCM_MACHINE: buzby2
...
SSTATE_CACHE: /sstate-cache/$TCM_MACHINE/PLAT3/
PURGE_SSTATE_CACHE: N
...
In my case /sstate-cache/$TCM_MACHINE/PLAT3/ is a directory in the docker container
that runs the build. This path is mounted in the docker container from a
permanent sstate cache directory on the build-server's filesystem, /var/bitbake/sstate-cache/<machine-id>/PLAT3.
PURGE_SSTATE_CACHE is overridable by a private variable
in the pipeline schedule settings so that I can optionally delete the cache for a squeaky clean
build.
Ensure that the setting of SSTATE_CACHE is appended to the bitbake conf/local.conf
file of the build, e.g.
.build_image: &build_image
stage: build
tags:
...
before_script:
...
script:
- echo "SSTATE_DIR ?= \"$SSTATE_CACHE\"" >> conf/local.conf
...
Apply the same pattern to DL_DIR if you use it.
Variables you use in the .yml file can be overriden by gitlab-ci trigger
or schedule variables. See Priority of variables