I'm using gitlab-runner to run CI/CD locally.
It works properly when I specify all jobs in .gitlab-ci.yml like
stages:
- test
test1:
stage: test
script:
- echo "ok"
and run gitlab-runner exec shell test1
In general, I'd like to store different jobs in different files. For example, I make test-pipeline.yml with jobs that relates to the test stage in the folder named .gitlab.
The .gitlab-ci.yml contains only to rows
include:
local: .gitlab/test-pipeline.yml
I commit and push changes to the remote repo and it works there but the command gitlab-runner exec shell job_name fails because it can't find such job.
Perhaps, I have to edit some of gitlab-runner's config but it's not obviously.
Has anybody faced with the same problem?
Thanks in advance!
gitlab-runner exec has many limitations. It does not have all the same features of the regular gitlab-runner. One such limitation is that it does not support the include: statement.
So, you won't be able to use gitlab-runner exec against this kind of config file that uses include:.
Related
I have two repositories in GitLab, repositories A and B let's say.
Repo A contains:
read_ci.yml
read_ci.sh
read_ci.yml contains:
stages:
- initialise
create checksum from pipeline:
stage: initialise
script:
- chmod +x read_ci.sh
- source ./read_ci.sh
Repo B contains:
gitlab-ci.yml
gitlab-ci.yml contains:
include:
project: 'Project/project_name'
file:
- '.gitlab-ci.yml'
ref: main
Obviously, this doesn't do what my intention is.
What I want to achieve is in the project B pipeline to run the project A script.
The reason is that I want project A to be called from multiple different pipelines and run there.
an alternative to this for GitLab: Azure Pipelines. Run script from resource repo
Submodules would absolutely work as Davide mentions, though it's kinda like using a sledgehammer to hang a picture. If all you want is a single script from the repository, just download it into your container. Use the v4 API with your CI_JOB_TOKEN to download the file, then simply run it using sh. If you have many files in your secondary repository and want access to them all, then use Submodules as Davide mentiones, and make sure your CI job retrieves them by setting the submodule strategy like this:
variables:
GIT_SUBMODULE_STRATEGY: normal
If you want to run the project A script in the project B pipeline, you can add the repository B as a git submodule in A
git submodule add -b <branch-B> <git-repository-B> <target-dir>
You need also to add in the CI job, the variable GIT_SUBMODULE_STRATEGY: recursive.
How I can call a specific runner for CI job. Problem: I have default runners and now I have to install my local runner and run tests on it. But I haven't a possibility to turn off runners on the Git because the whole project builds on remote drivers.
You can use tags to tag a runner, when you register it. And you can specify that your job will only run on runners with this tags.
eg. you tagged your runner with 'fancy-example' than you can use it like
build:
tags:
- fancy-example
script:
- echo Hello
See the gitLab docs for more detailed examples and explanations:
https://docs.gitlab.com/ce/ci/yaml/README.html#tags
Help, please. I have problems when using the CI tool.
Here's my .gitlab-ci.yaml
stages:
- test
test:
stage: test
environment:
name: test
url: https://word.mymusise.com/env_test.txt
script: echo "Running tests TEST=$TEST"
And I've define the test environment in EnvDocker > Pipelines > Environments
But it didn't export the environment from https://word.mymusise.com/env_test.txt in the CI job.
Running with gitlab-runner 11.4.2 (cf91d5e1)
on gitlab-ci runner a0e18516
Using Docker executor with image ubuntu:16.04 ...
Pulling docker image ubuntu:16.04 ...
Using docker image sha256:2a697363a8709093834e852b26bedb1d85b316c613120720fea9524f0e98e4a2 for ubuntu:16.04 ...
Running on runner-a0e18516-project-123-concurrent-0 via gitlab...
Fetching changes...
HEAD is now at d12c05b Update .gitlab-ci.yml
From https://gitlab.kingdomai.com/mymusise/envdocker
d12c05b..1a3954f master -> origin/master
Checking out 1a3954f8 as master...
Skipping Git submodules setup
$ echo "Running tests TEST=$TEST"
Running tests TEST=
Job succeeded
I define export TEST="test" in https://word.mymusise.com/env_test.txt, but it seems not working.
What should I do... Orz
Gitlab version: 11.4.0-ee
You want to run commands that are inside the text file that is accessible via http protocol.
With curl you can download the file and print it on curl's standard output. With command substitution $() you can grab the standard output. Then you can execute the commands itself (very unsafe, there might be multiple escaping issues).
script:
- $(curl "$url")
- echo "Running tests TEST=$TEST"
A safer alternative would be to just download the file and execute/source it.
script:
- curl "$url" > ./run_this.sh
# don't forget to add executable right to the file ;)
- chmod +x ./run_this.sh
- source ./run_this.sh
# pick out the trash
- rm ./run_this.sh
# rest of your script.
- echo "Running tests TEST=$TEST"
Downloading a shell script and executing it is a popular way of automating tasks, usually with curl url | bash. It is not supported "natively" by gitlab and I don't think it should be.
GitLab-CI executes the stop-environment script in dynamic environments after the branch has been deleted. This effectively forces you to put all the teardown logic into the .gitlab-ci.yml instead of a script that .gitlab-ci.yml just calls.
Does anyone know a workaround for this? I have a shell script that removes the deployment. This script is part of the repository and can also be called locally (i.e. not onli in an CI environment). I want GitLab-CI to call this script when removing a dynamic environment but it's obviously not there anymore when the branch has been deleted. I also cannot put this script to the artifacts as it is generated before the build by a configure script and contains secrets. It would be great if one could execute the teardown script before the branch is deleted.
Here's a relevant excerpt from the .gitlab-ci.yml
deploy_dynamic_staging:
stage: deploy
variables:
SERVICE_NAME: foo-service-$CI_BUILD_REF_SLUG
script:
- ./configure
- make deploy.staging
environment:
name: staging/$CI_BUILD_REF_SLUG
on_stop: stop_dynamic_staging
except:
- master
stop_dynamic_staging:
stage: deploy
variables:
GIT_STRATEGY: none
script:
- make teardown # <- this fails
when: manual
environment:
name: staging/$CI_BUILD_REF_SLUG
action: stop
Probably not ideal, but you can curl the script using the gitlab API before running it:
curl \
-X GET https://gitlab.example. com/raw/master/script.sh\
-H 'PRIVATE-TOKEN: ${GITLAB_TOKEN}' > script.sh
GitLab-CI executes the stop-environment script in dynamic environments after the branch has been deleted.
That includes:
An on_stop action, if defined, is executed.
With GitLab 15.1 (June 2022), you can skip that on_top action:
Force stop an environment
In 15.1, we added a force option to the stop environment API call.
This allows you to delete an active environment without running the specified on_stop jobs in cases where running these defined actions is not desired.
See Documentation and Issue.
I have a circle.yml file that looks something like this:
general:
branches:
only:
- master
- develop
- /release-[0-9]+(\.[0-9]+)*/
test:
pre:
- docker-compose run $SERVICE npm install
override:
- docker-compose run $SERVICE npm test
post:
- docker-compose run SPECIFIC_COMMAND // this should only trigger for branches that fall under /release-[0-9]+(\.[0-9]+)*/
- docker-compose stop
Unit tests run when merging to master, develop, or /release-[0-9]+(\.[0-9]+)*/.
However, there's a certain command in the post-hook under tests that I would only like to trigger when merging into /release-[0-9]+(\.[0-9]+)*/. This command must run before the final one, docker-compose stop, which is why I haven't used a deployment block.
Turns out this isn't quite possible in the test block (unlike the branches or deployment blocks).
The best way around this was to place the conditional logic in a shell script that accessed the $CIRCLE_BRANCH environment variable. The shell script in turn would be triggered all the time.