Running gradle release on gitlab pipeline triggers 3 more build jobs - gitlab

This is my gitlab pipeline:
image: docker:latest
services:
- docker:dind
variables:
DOCKER_DRIVER: overlay
SPRING_PROFILES_ACTIVE: gitlab-ci
before_script:
- git remote set-url origin https://gitlab-ci-token:$CICD_GIT_TOKEN#$CI_SERVER_HOST/$CI_PROJECT_PATH.git
- git config --global user.email 'me#email.com'
- git config --global user.name 'Gitlab (Me)'
- git checkout -b $CI_COMMIT_BRANCH
stages:
- build
- package
- deploy
gradle-build:
image: gradle:alpine
stage: build
script:
- PROJECT_VERSION=`./gradlew -quiet printVersion`
- ./gradlew release
artifacts:
paths:
- build/libs/*.jar
docker-build:
stage: package
script:
- docker build -t registry.gitlab.com/me/myproject:$PROJECT_VERSION .
- docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN registry.gitlab.com
- docker push registry.gitlab.com/me/myproject:PROJECT_VERSION
Not sure why the 3 builds get triggered (maybe because the plugin does a commit) and not sure how to fix this.
Any help would be appreciated.
Thanks.

After taking a look here:
https://docs.gitlab.com/ee/ci/yaml/
I added the following section to the pipeline:
workflow:
rules:
- if: $CI_COMMIT_AUTHOR =~ /^Gitlab Pipeline/
when: never
- when: always
Moved the stuff in "before_script" under script, and changed the git username:
script:
- git remote set-url origin https://gitlab-ci-token:$CICD_GIT_TOKEN#$CI_SERVER_HOST/$CI_PROJECT_PATH.git
- git config --global user.email 'me#email.com'
- git config --global user.name 'Gitlab Pipeline'
- git checkout -b $CI_COMMIT_BRANCH
- ./gradlew release
Now the commits with the name "Gitlab Pipeline" don't trigger a build anymore.
The extra builds were triggered because the release plugin also does a pre-tag commit, followed by a new version commit at the end. This would trigger 2 builds, which in turn would each trigger 2 more and so on.
Now the pipeline behaves as expected.

Related

Merge Gitlab Runner into existing branch

I'm right now on a stage of automating some parts of our code. The idea is simple, with every commit in our gitlab the script takes Excel from given directory and divides it's worksheets into individual Excel files and saves them into another directory. The problem right now is that I want to merge those files into our branch from Gitlab Runner. I tried editing gitlab-ci file, but I can't get it to work. I get this error:
The request URL returned error: 403
I tried adding personal token and then using it for pushing like this:
variables:
GIT_STRATEGY: clone
build-job:
stage: build
tags:
- data_dict
script:
- echo "Hello!"
- ls
- python3 -V
- pip3 list
test-job1:
stage: test
tags:
- data_dict
script:
- python3 Experiments/ConfAutomation.py
test-job2:
stage: test
tags:
- data_dict
script:
- git show-ref
- git remote -v
- echo "$RUNNER_ACCESS_TOKEN"
- echo "Print runner branch name"
- git config user.email "user#mail.com"
- git config user.name "name"
# - git remote set-url --push origin https://gitlab-ci:"$RUNNER_ACCESS_TOKEN"#gitlab/dir.git
- git add .
- git commit --allow-empty -m "Files from runner to branch"
- git push origin https://gitlab-ci:"$RUNNER_ACCESS_TOKEN"#gitlab/dir.git <branch-name>
deploy-prod:
stage: deploy
tags:
- data_dict
script:
- echo "This job deploys something."
I tried some stuff but can't seem to get it to work. Maybe you had similiar problem or you have some ideas ? Any help would be greatly appreciated.
Try using {$RUNNER_ACCESS_TOKEN} without double quotes.

Pushing tags triggers CI/CD loop

Im using CircleCI for CI/CD and recently wanted to start tagging my releases to main bransch.
This is my config.yml
version: 2.1
orbs:
node: circleci/node#1.1.6
jobs:
build:
executor:
name: node/default
steps:
- checkout
- node/with-cache:
steps:
- run: git pull
- run: npm install standard-version
- run: npm run release
- run: git push --follow-tags origin master
- node/with-cache:
steps:
- run: echo 'deploying master branch'
Ofcourse this triggers and endless loop since it creates a new push which triggers CircleCI... I've read that you can skip builds in commit messages by adding [ci skip] but pushing tags does not give that option.
How do I get around this? I want the semantic releases to be tagged automatically when a new version is released... Can I somehow get around this?
Im using standard-version for tagging and updating package.json.
Any help would be greatly appriciated
The solution was the following changes to my config.yml:
version: 2.1
orbs:
node: circleci/node#1.1.6
jobs:
build:
executor:
name: node/default
steps:
- checkout
- node/with-cache:
steps:
- checkout
- run: npm install
- run: npm run test
- run: npx semantic-release
- node/with-cache:
steps:
- run: echo 'deploying master branch'
- run: ssh -v -o "StrictHostKeyChecking no" user#xxxx.xxx "cd ~/projects/xxx; git pull --rebase; ./publish.sh"
workflows:
version: 2
build_project:
jobs:
- build:
filters:
branches:
only: master
When doing this with "- run: npx semantic-release", the version bumping step will be auto incremented and the commit tagged with [skip ci] so it doesnt loop.

Automatic code merging using gitlab-ci.yml pipeline

My goal is to create Gitlab yml file in order to automatically merge code from release to master branch which will be run on a schedule (once a week)
Here is my .gitlab-ci.yml:
image: "python:3.8"
stages:
- build
- test
- release
Build:
stage: build
script:
- <some code here>
Test:
stage: test
script:
- <some code here>
job:on-schedule:
stage: release
only:
- schedules
script:
- git config --global user.email myEmail#gmail.com
- git config --global user.name "myUserName"
- git checkout release
- git checkout master
- git merge release
But I'm getting the following error:
error: pathspec 'release' did not match any file(s) known to git
Cleaning up file based variables
00:00
ERROR: Job failed: exit code 1
. I'd appreciate it if you have any ideas on how to fix that.
By default, gitlab-ci does a shallow clone, and so the error you're seeing of:
error: pathspec 'release' did not match any file(s) known to git
is accurate because the release branch does not exist locally on the runner.
To resolve this, you can just update the lines to:
- git checkout origin/release
- git checkout origin/master
which will pull the latest branches from the repository directly instead.

How to trigger Gitlab CI pipeline manually, when in normal conditions, it is triggered by webhook with commit Ids?

I have Gitlab CI pipeline which is triggered by bitbucket webhook with current and last commit ids. I also want to re-run pipeline manually whenever the build created Gitlab CI file, triggered by webhook is not working as expected.
I tried RUN-PIPELINE option but shows the error:
The form contains the following error:
No stages/jobs for this pipeline.
Here is the GitLab CI file. Include refers to other project where standard yaml file for the pipeline is kept:
include:
- project: Path/to/project
ref: bb-deployment
file: /bitbucket-deployment.yaml
variables:
TILLER_NAMESPACE: <namespace>
NAMESPACE: testenv
REPO_REF: testenvbranch
LastCommitSHA: <commit sha from webhook>
CurrentCommitSHA: <Current commit she from webhook>
Here is the detailed gitlab-ci file that is provided in other project which has stages:
stages:
- pipeline
- build
variables:
ORG: test
APP_NAME: $CI_PROJECT_NAME
before_script:
- 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
- eval $(ssh-agent -s)
- echo "$SSH_PRIIVATE_KEY2" | tr -d '\r' | ssh-add -
- mkdir -p ~/.ssh
- chmod 700 ~/.ssh
- echo "$SSH_KNOWN_HOSTS" > ~/.ssh/known_hosts
- chmod 644 ~/.ssh/known_hosts
Building CI Script:
stage: pipeline
image: python:3.6
only:
refs:
- master
script:
- |
curl https://github.com/org/scripts/branch/install.sh | bash -s latest
source /usr/local/bin/pipeline-variables.sh
git clone git#bitbucket.org:$ORG/$APP_NAME.git
cd $APP_NAME
git checkout $lastCommit
cp -r env old
git checkout $bitbucketCommit
$CMD_DIFF old env
$CMD_BUILD
$CMD_INSTALL updatedReposList.yaml deletedReposList.yaml /tmp/test $NAMESPACE $REPO_REF $ORG $APP_NAME $lastCommit $bitbucketCommit
cat cicd.yaml
mv cicd.yaml ..
artifacts:
paths:
- cicd.yaml
Deplopying Apps:
stage: build
only:
refs:
- master
trigger:
include:
artifact: cicd.yaml
job: Building CI Script
strategy: depend
In the manual trigger, instead of considering the last and current commit she, it should rebuild the application.
Any help will be appreciated.
Thank you for your comment (below), I see you are using the include directive (https://docs.gitlab.com/ce/ci/yaml/#include) in one .gitlab-ci.yml to include a GitLab CI YAML file from another project.
I can duplicate this error (No stages / jobs for this pipeline) by invoking "run pipeline" on project 1 which is configured to include GitLab CI YAML from project 2 when the project 2 GitLab CI YAML is restricted to the master branch but I'm running the project on another branch.
For example, let's say project 1 is called "stackoverflow-test" and its .gitlab-ci.yml is:
include:
- project: atsaloli/test
file: /.gitlab-ci.yml
ref: mybranch
And project 2 is called "test" (in my own namespace, atsaloli) and its .gitlab-ci.yml is:
my_job:
script: echo hello world
image: alpine
only:
refs:
- master
If I select "Run Pipeline" in the GitLab UI in project 1 on a branch other than "master", I then get the error message "No stages / jobs for this pipeline".
That's because there is no job defined for my non-master branch, and then without any job defined, I don't have any stage defined.
I hope that sheds some light on what's going on with your webhook.

How to disable fetching git repository by gitlab runner?

There is such task in my .gitlab-ci.yml
deploy_all:
stage: deploy
script:
- cp project/target/jnlp/* html/jnlp/
tags:
- client:deploy-all
Everything works fine except unnecessary git repository fetching. Below is extract from runner's log
Running with gitlab-ci-multi-runner 9.1.0 (0118d89)
...
Fetching changes...
HEAD is now at 8dfc104 Update .gitlab-ci.yml
...
Job succeeded
The repository is not needed here because I need only artifacts from other tasks. Is it possible to disable this behaviour?
I found the solution:
upgraded gitlab to version 10.x, manual is here https://docs.gitlab.com/runner/install/linux-repository.html
disabled git checkout in the build script (by adding variables)
deploy_all:
variables:
GIT_STRATEGY: none
GIT_CHECKOUT: "false"
stage: deploy
script:
- cp project/target/jnlp/* html/jnlp/
tags:
- client:deploy-all

Resources