I'm trying to create a pipeline on Gitlab CI that increments the app version everytime we get a commit on master. But it is ignoring my ci.skip command and I don't know why.
The yaml file is this one:
.gitlab-ci.yml
workflow:
rules:
- if: $CI_COMMIT_BRANCH == 'master'
before_script:
- git config --global user.email "${GITLAB_USER_EMAIL}"
- git config --global user.name "${GITLAB_USER_NAME}"
- git remote set-url origin https://push:$PUSH_KEY#$CI_SERVER_HOST/$CI_PROJECT_PATH.git
auto_release:
image: node:10
script:
- yarn
- yarn release
- git push --follow-tags origin HEAD:master -o ci.skip
- echo "Done!"
So everytime I push a new commit it gets locked inside an eternal loop that commits a new version and commits a new version over and over again. The only way to stop is manually cancelling the jobs.
Pleas note: When we use the image node or node:latest it works, but our version requires node:10 otherwise it will break and won't build.
node:10 is a very old image. The git version it contains does not support push options (at least with the shorthand -o), so that's why the push triggers the next CI build.
Check the git version in the image - if it's 2.10 to 2.17 you can use --push-option=ci.skip. If it's still an older version, you need to create your own docker image that contains node version 10 and a modern git version.
I created a pipeline to deploy my code from Bitbucket repo to Heroku server.
my bitbucket-pipelines.yml
image: node:10.15.3
pipelines:
default:
- step:
script: # Modify the commands below to build your repository.
- npm install
- git push https://heroku:<my heroku api key>#git.heroku.com/<my heroku app's name>.git HEAD
After pushing my code to my bitbucket repo, the pipeline runs, but fails while pushing the code to heroku git.
Below is the error:
Push rejected, source repository is a shallow clone. Unshallow it with `git fetch --all --unshallow` and try pushing again.
But then by bit bucket repo is not shallow. The command git rev-parse --is-shallow-repository returns false.
You can use the following git command before the push
git filter-branch -- --all
So it would be in your script:
...
- git filter-branch -- --all
- git push https://heroku:<my heroku api key>#git.heroku.com/<my heroku app's name>.git HEAD
I have several projects, each in their own repository, that import a common library which has its own repository as well.
So, the .gitmodules file includes the library with the full name:
Submodule 'xx/yy' (https://gitlab.com/xx/yy.git) registered for path 'xx/yy'
but this doesn't work:
Fatal: could not read Username for 'https://gitlab.com': No such device or address
the CI script is very simple:
image: mcr.microsoft.com/dotnet/core/sdk:3.0.100-preview9-alpine3.9
variables:
GIT_SUBMODULE_STRATEGY: recursive
stages:
- build
before_script:
- "cd xx"
- "dotnet restore"
build:
stage: build
script:
- "cd xx"
- "dotnet build"
The old answer was:
GitLab pull submodules inside CI
but things have changed and we can, according to the docs, have submodules that don't have a relative path, as written here: https://docs.gitlab.com/ce/ci/git_submodules.html
tldr; like this:
# .gitlab-ci.yml
stages:
- build
job1:
stage: build
before_script:
- git config --global credential.helper store
- git config --global credential.useHttpPath true
- |
git credential approve <<EOF
protocol=https
host=gitlab.com
path=my-group/my-submodule-repo.git
username=${CI_DEPENDENCY_PROXY_USER}
password=${CI_DEPENDENCY_PROXY_PASSWORD}
EOF
- git submodule update --init --recursive
script:
- echo "Let's start the build..."
Explanation
The stages: - build and job1: stage: build declarations are boilerplate --- they inform the gitlab ci machinery that there exists one stage (named build) and one job that "belongs" to this stage.
The before_script part details things that need to happen early in the job --- everything thereunder must complete before script is started.
The git config --global credentials.helper tells git to use the credentials helper named "store". By default, this is a cleartext file located at ~/.git-credentials containing newline-delimited username-password-decorated URIs, each corresponding to a given git remote added by the user.
The git config --global credentials.useHttpPath tells git not to ignore the path attribute for any call (explicit or otherwise) to git credential. This is not strictly necessary, but rather good practice when, for example, you have multiple git remotes on the same host.
The git credential approve reads standard input (expressed as a heredoc) and passes the given credential to the credential.helper, namely store, to be written into ~/.git-credentials.
The git submodule update --init --recursive populates the existing (but as yet incomplete) superproject worktree with the content referenced by .gitmodules.
Assumptions
This aforementioned example makes the following assumptions:
The superproject .gitmodules contains a reference to the submodule remote https://gitlab.com/my-group/my-submodule-repo.git.
The git submodule remote is private; i.e., access thereto requires credentials in the form of a username and password ("personal access token" in gitlab parlance)
You want to authenticate to this remote using the Gitlab CI Dependency Proxy credentials described here: https://about.gitlab.com/blog/2020/12/15/dependency-proxy-updates/ .
You don't want to re-write the .gitmodules to use relative URLs as suggested here: https://docs.gitlab.com/ee/ci/git_submodules.html#configure-the-gitmodules-file .
References:
https://docs.gitlab.com/ee/ci/yaml/#before_script
https://docs.gitlab.com/ee/ci/git_submodules.html#using-git-submodules-in-your-ci-jobs
https://docs.gitlab.com/ee/ci/yaml/README.html#git-submodule-strategy
https://gitlab.com/gitlab-org/gitlab/-/issues/208770
https://git-scm.com/docs/gitcredentials
https://git-scm.com/docs/git-credential-store
https://git-scm.com/docs/git-credential
I need my Gitlab CI to update submodules with --remote flag so that the HEAD is set to the remote's HEAD. After a bit of Googling I found that I need to set GIT_SUBMODULE_STRATEGY to none and run git submodule update --recursive --remote --init manually:
variables:
GIT_STRATEGY: clone
GIT_SUBMODULE_STRATEGY: none
before_script:
- apk add git || ( apt-get update && apt-get -y install git )
- git submodule update --recursive --remote --init
test:build:
services:
- docker:dind
image: ubuntu
variables:
DOCKER_HOST: tcp://docker:2375
DOCKER_DRIVER: overlay2
script:
- echo "done
Unfortunately I'm getting a CI failure (names edited):
$ git submodule update --recursive --remote --init
Submodule 'current_project_name/submodule_project_name' (ssh://git#gitlab.someserver.net:9931/someorg/submodule_project_name.git) registered for path 'current_project_name/submodule_project_name'
Cloning into '/builds/someorg/current_project_name/current_project_name/submodule_project_name'...
Host key verification failed.
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
fatal: clone of 'ssh://git#gitlab.someserver.net:9931/someorg/submodule_project_name.git' into submodule path '/builds/someorg/current_project_name/current_project_name/submodule_project_name' failed
Failed to clone 'current_project_name/submodule_project_name'. Retry scheduled
Cloning into '/builds/someorg/current_project_name/current_project_name/submodule_project_name'...
Host key verification failed.
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
fatal: clone of 'ssh://git#gitlab.someserver.net:9931/someorg/submodule_project_name.git' into submodule path '/builds/someorg/current_project_name/current_project_name/submodule_project_name' failed
Failed to clone 'current_project_name/submodule_project_name' a second time, aborting
I can see that the CI does have permissions to clone that submodule_project_name because if I set GIT_SUBMODULE_STRATEGY e.g. to recursive, CI manages to pull it (but it's not --remote, so it doesn't work the way I want). Unfortunately when my before_script tries to do it, I'm getting the error. How can I bypass it?
I mentioned before updating the ~/.ssh/.known_hosts file, as in here.
This is not needed when fetching the submodules before the script (which is not what you are doing with GIT_SUBMODULE_STRATEGY set to NONE)
With dind (Docker In Docker), consider also this thread, regarding ssh-add for private keys, and .dockerini / .dockerenv SSH directives.
The OP d33tah confirms in the comments:
I actually didn't add any key, assuming that since Gitlab CI's defaults can pull the key, I should be able to as well.
Then I found that docs say that I needed a deploy key and I added one
Yes: adding the public key on Gitlab side is mandatory.
In my CI pipeline I am generating an artifact public/graph.png that visualises some aspect of my code. In a later step I want to commit that to the repo from within the CI pipeline. Here's the pertinent part of .gitlab-ci.yml:
commit-graph:
stage: pages
script:
- git config user.email "cipipeline#example.com"
- git config user.name "CI Pipeline"
- cd /group/project
- mv public/graph.png .
- git add graph.png
- git commit -m "committing graph.png [ci skip]"
- echo $CI_COMMIT_REF_NAME
- git push origin HEAD:$CI_COMMIT_REF_NAME
When the pipeline runs within gitlab it fails with:
$ git config user.email "cipipeline#dhgitlab.dunnhumby.co.uk"
$ git config user.name "CI Pipeline"
$ cd /group/project
$ mv public/graph.png .
$ git add graph.png
$ git commit -m "committing graph.png [ci skip]"
[detached HEAD 22a50d1] committing graph.png [ci skip]
1 file changed, 0 insertions(+), 0 deletions(-)
create mode 100644 graph.png
$ echo $CI_COMMIT_REF_NAME
jamiet/my-branch
$ git push origin HEAD:$CI_COMMIT_REF_NAME
fatal: unable to access 'https://gitlab-ci-token:xxxxxxxxxxxxxxxxxxxx#example.com/group/project/project.git/': server certificate verification failed. CAfile: /etc/ssl/certs/ca-certificates.crt CRLfile: none
Not sure what I'm doing wrong and don't know enough about SSL to understand that error. Can anyone advise?
We are hosting gitlab ourselves by the way.
Nowadays there is a much cleaner way to solve this without using SSH but using a project scoped access token, also see this answer.
In the GitLab project create an project scoped access token so it is linked to the project, not to an individual. Next store this token as an GitLab CI/CD variable. You can now connect using the following:
push-back-to-remote:
script:
- git config user.email "my-email#email.com"
- git config user.name "ci-bot"
- git remote add gitlab_origin https://oauth2:$ACCESS_TOKEN#gitlab.com/path-to-project.git
- git add .
- git commit -m "push back from pipeline"
- git push gitlab_origin HEAD:main -o ci.skip # prevent triggering pipeline again
Some finger push-ups still required, but here's a less brittle way of pushing to the repository from its own CI, that I use in my daily work. It pushes to master directly from a detached head:
Generate an RSA key and add it as a Project Deploy Key with write access (the public part).
Put the private part into your CI/CD variables from inside of your project settings as SSH_PUSH_KEY. Make sure to set it as protected.
Add a CI_KNOWN_HOSTS variable, with the SSH fingerprint of your GitLab instance (remember that thing ssh asks you about the first time you try to connect to a host? That.).
Use ssh-keyscan <gitlab-host> to get it. It will look similar to this:
my.gitlab.instance.com ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEArlUMUmNj59PpoLyy4EsKbwhPUfXxuAzFN7dMKDXVvKMmN8344HqQV1tRx6fcmH+0BXK1JAP4f10V0VnYti3e1c5f9dhpl8pIqKLMJgdGDq3MLqjihL3bp5xm8nDsPTm5FoEPPYK1I3M2wr18pBB19evz64NHrK5R/HO5LyTrybVasFumt8cZoH6crnCFgfQWV1mHAG3j41Q0z4yxu6g8zBWESZcVVn90HxQH7+LDHx11122233344491MQGl5fZcKqVWsWQVEssaK87iBsWUxvsuoeVUrj4YRcmbi6F4+ZZZZZZZwwww3ZboWsSWxTk5ESR6WWHccBm8GQflXyY3ZQ==
Set up your job inside .gitlab-ci.yml as follows. Set stage and resource_group options appropriately - without the latter you might run into race conditions. Also, make sure to set only properly, as otherwise your CI might trigger itself:
"This CI job pushes to its own repo":
stage: my_push_stage
resource_group: this_option_comes_handy_when_pushing
only:
- triggers
before_script:
- mkdir ~/.ssh/
- echo "${CI_KNOWN_HOSTS}" > ~/.ssh/known_hosts
- echo "${SSH_PUSH_KEY}" > ~/.ssh/id_rsa
- chmod 600 ~/.ssh/id_rsa
- git config user.email "ci#example.com"
- git config user.name "CI"
- git remote remove ssh_origin || true # Local repo state may be cached
- git remote add ssh_origin "git#$CI_SERVER_HOST:$CI_PROJECT_PATH.git"
script:
- touch "xyz" # Make an edit
- git add "xyz"
- git commit -m "My CI commit"
- git push ssh_origin HEAD:master # ❗ this pushes to master,
# use $CI_COMMIT_REF_NAME if you want to push to current branch
- git tag MyCiTag # If you need to add a tag you can do that too
- git push --tags ssh_origin
TL;DR;
If you just want to make a release or version bump commits when stuff is merged to main, there are probably existing CLI tools that you can use in a much more simpler and well documented way.
(See Use the gitlab api for an example)
How to do it
The necessary steps:
commitingJob:
stage: some_stage
script:
# in order to commit we have to have a user set
# this would also make it easy to distinct the CI-made commits
- git config user.name "CI Pipeline"
- git config user.email "cipipeline#example.com"
# stage some changes
- git add src/*
# We can use `-o ci.skip`, but AFAIK it doesn't work for Merge Request pipelines, while having it inside the commit message works there as well
- git commit -m "A commit message [skip ci]"
# we're on a detached head but we can push the commits we made to the remote branch like so:
- git push HEAD:$CI_COMMIT_REF_NAME
One thing missing here is authenticating with the remote. It can be setup in a number of ways (ssh, token, api). Other answers discuss a few different options. My preference would be to use an ACCESS_TOKEN, see how and why bellow
Summary and critique on the available authentication options
Adding or copying an ssh key inside the CI steps
Compared to just using an access token, going for the ssh way seems like too much work, for the same result
generate an ssh key
save the public key as Project deploy token
save the private key as a Project CI variable
dedicate a few CI steps on
adding the private keys
adding known hosts
add/update the ssh remote
Overall it seems like something you'd do for a user and not for a pipeline
You have to add the private key as a Project Level CI variable - due to pattern restrictions the variable can't be masked. It can be added as a file, but then you'd have to chmod for proper permissions...
The positive thing I see here is the ssh key is restricted only to the projects you add it to as deploy token
The easiest option seems to be to use an ACCESS_TOKEN
Using a token to push commits is as easy as updating the push remote like this:
git remote set-url --push origin "https://$TOKEN_NAME:$ACCESS_TOKEN#gitlab.com/<project>.git"
TOKEN_NAME - the name you set for the token in gitlab (preferably pick a name without space characters)
ACCESS_TOKEN - this can be a personal access token (Free Tier) or a project access token (Premium Tier)
It would have to be stored as Project Level CI variable, and it can be masked.
I would just save a variable like CI_COMMITTER_USER_AND_TOKEN containing both <token_name>:<access_token> content
⚠ Warning about Personal Access Tokens
This is too broad permission to give to a CI Runner even if the runner should commit to multiple repositories, I'll prefer separate access tokens per project with only the minimum permissions needed
Keep in mind that if someone takes hold of your personal access token, they can do stuff (like committing in this case) on your behalf. Giving the write_repository permission means you can use the token to write to any repository you're account has access too.
CI_JOB_TOKEN
If you're wondering whether you can use the CI_JOB_TOKEN to push commits - you can't. You can only pull with that thing
(You can also use it to publish packages and images, but not push commits)
Why git remote set-url --push origin
We can add a separate remote for CI made commits, but keep in mind it might get cached, so if we just do
script:
- git remote add ci "https://$TOKEN_NAME:$ACCESS_TOKEN#gitlab.com/<project>.git"
the next time the pipeline runs we'll get an error from git remote add, because the remote we're trying to add is already added
set-url on the existing origin makes sure you use the latest url
To achieve the same with git remote add we'd have to first try to remove the ci remote and then add it back
script:
- git remote remove ci || true
- git remote add ci "https://$TOKEN_NAME:$ACCESS_TOKEN#gitlab.com/<project>.git"
Use the gitlab api to push changes
This works pretty much the same way as the ACCESS_TOKEN way, but instead of the write_repository the token has the api permission
If the reason to commit and push as part of CI is releasing a new version on merge to main then using CLI tools and a guide like this might be a better option than setting up the committing and pushing yourself: https://docs.gitlab.com/ee/ci/examples/semantic-release.html
The above guide setups a version bump, package updates, and release notes.
Note how this step instructs us to create an Access Token with api permission in order for the underlying CLIs to commit the changes: https://docs.gitlab.com/ee/ci/examples/semantic-release.html#set-up-cicd-variables
You can add the CI_SERVER_CLS_CA_FILE to sslCAInfo git config.
checkout alchemy:
stage: prepare
script:
- git config --global "http.${CI_SERVER_URL}.sslCAInfo" "$CI_SERVER_TLS_CA_FILE"
- git clone https://gitlab-ci-token:${CI_JOB_TOKEN}#${CI_SERVER_HOST}/sparklemuffin/alchemy.git
#Sjoerd's approach, in the comments, to export GIT_SSL_CAINFO instead, is a bit shorter.
export GIT_SSL_CAINFO=$CI_SERVER_TLS_CA_FILE
Gitlab creates CI_SERVER_TLS_CA_FILE and configures git to use it for initially cloning the repository. For some reason this configuration is no longer available later on.
I found this GitLab forum link helpful
As suggested by the user you need to generate SSH key, associate it with new GitLab user dedicated for this job and add key to the runner. Small drawback is you need to use swap origin in gitlab for original ssh source (instead of sandboxed one used inside the job) which leads to committer being changed to mentioned new account instead of person who triggered pipeline.
Source from link:
# for your information
whoami
printenv
# we need to extract the ssh/git URL as the runner uses a tokenized URL
export CI_PUSH_REPO=`echo $CI_REPOSITORY_URL | perl -pe 's#.*#(.+?(\:\d+)?)/#git#\1:#'`
# runner runs on a detached HEAD, create a temporary local branch for editing
git checkout -b ci_processing
git config --global user.name "My Runner"
git config --global user.email "runner#gitlab.example.org"
git remote set-url --push origin "${CI_PUSH_REPO}"
# make your changes
touch test.txt
# push changes
# always return true so that the build does not fail if there are no changes
git push origin ci_processing:${CI_COMMIT_REF_NAME} || true
Just with current version of GitLab you need to change source variable name as follows:
export CI_PUSH_REPO=`echo $CI_REPOSITORY_URL | perl -pe 's#.*#(.+?(\:\d+)?)/#git#\1:#'`
Instead of declaring CI_KNOWN_HOST, you can try get it realtime:
- 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
- ssh-keyscan -t rsa $CI_SERVER_HOST >> ~/.ssh/known_hosts
Solved it. Issuing git config --global http.sslverify "false" prior to the push solved that particular problem (it exposed another problem but that's for another thread :) )
I can commit from Gitlab-CI with a selected user with a minor change based on tsr's answer https://stackoverflow.com/a/57800614/5269825 :
# set remote URL to https://oauth2:<AccessToken>#server.com/project.git
CI_PUSH_REPO=`echo "$CI_REPOSITORY_URL $ACCESS_TOKEN_PARAM" | sed 's/^.*\(#.*\)\s\(.*\)/https:\/\/oauth2:\2\1/g'`
git config http.sslverify false
git remote set-url --push origin "${CI_PUSH_REPO}"
git config user.name "Token Owner"
git config user.email "tokenowner#email.com"
# runner runs on a detached HEAD, create a temporary local branch for editing
git checkout -b ci_processing
# make your changes
# push changes
# always return true so that the build does not fail if there are no changes
git push origin ci_processing:${CI_COMMIT_REF_NAME} || true
The ACCESS_TOKEN_PARAM must be configured at the project's CI/CD Variables configuration.
The idea of using Oauth2 and Access Token was taken from https://stackoverflow.com/a/52074198/5269825 and https://stackoverflow.com/a/52154378/5269825.
Also, pushing changes can trigger a new pipeline!
In my case deploy keys option was optimal (compared to personal tokens or CI token - only supports basic auth) with Gitlab Shell Runner. In case someone is struggling with pushing to from Gitlab CI, this can be done sharing the public key of the runner with the Gitlab server
This is a working example as of today
environment: k8s
gitlab: 13.x
gitlab runner: 13.x
goal of this job named convert:- Converts the excel files to json files and commits and updates the branch of the repo.
convert:
variables:
REPO: "gitlab.com/group/myproject.git" # example
BRANCH: "BRANCHNAME" # example
# recommended to store the following as project variables to hide secrets away from the gitlab ci file.
GITLAB_USER_ID: "gitlab_user" # example
CI_USERNAME: "gitlab_user" # example
CI_PUSH_TOKEN: "<api token from gitlab" # example
GITLAB_USER_EMAIL: "gitlab_user#company.com" # example
stage: convert
image:
name: python:3.7-buster
entrypoint: ["/bin/ash"]
before_script:
- pip3 install openpyxl
- ls -altr
script:
- echo 'converting excel to json'
- python excel2json.py
- git remote set-url origin https://${CI_USERNAME}:${CI_PUSH_TOKEN}#$REPO
- git config --global user.email '${GITLAB_USER_EMAIL}'
- git config --global user.name '${GITLAB_USER_ID}'
- git add -A && git commit -m 'added/updated json files'
- git push origin HEAD:$BRANCH
Note: CI_USERNAME == GITLAB_USER_ID. Both are same in my case.
None of these worked immediately for me but I merged them all together to come up with this:
- git config user.email "example#example.com"
- git config user.name "example"
- git remote remove dev || true
- git remote add dev https://example:$PROJECT_ACCESS_TOKEN#gitlab.com/demo/versioning-test.git
- git remote set-url --push origin "https://example:$PROJECT_ACCESS_TOKEN#gitlab.com/demo/versioning-test.git"
- git pull origin dev
- git checkout dev
- git reset --hard origin/dev
- git rm build_number.json || true
#grabs the build number from tags and pipes it into text file json formatted
- echo "`echo {'"build_number"':` `git describe --tags` `echo }`" | sed -e 's/\s\+/"/g' >> build_number.json
- git add build_number.json
- git commit -m "push back from pipeline [skip ci]"
- git push origin dev
My use case was that we wanted to bump to a version file and push that back into the repo.
Another way:
Create dedicated Gitlab account
Add CI/CD variable GIT_CICDUSER_PASSWORD containing it's password to the project inside the main account running the pipeline
Then a job can look like:
task_requiring_a_push:
stage: some-stage
variables:
GIT_STRATEGY: none
script:
- git config --global user.email "user#mail.com"
- git config --global user.name "CI/CD User"
- git clone https://<dedicated_gitlab_user_username>:$GIT_CICDUSER_PASSWORD#gitlab.com/$CI_PROJECT_PATH .
- git checkout $CI_COMMIT_REF_NAME
- # do something
- git add *
- git commit -m "message"
- git push --push-option=ci.skip origin $CI_COMMIT_REF_NAME
only:
- release-*