Merge Gitlab Runner into existing branch - gitlab

I'm right now on a stage of automating some parts of our code. The idea is simple, with every commit in our gitlab the script takes Excel from given directory and divides it's worksheets into individual Excel files and saves them into another directory. The problem right now is that I want to merge those files into our branch from Gitlab Runner. I tried editing gitlab-ci file, but I can't get it to work. I get this error:
The request URL returned error: 403
I tried adding personal token and then using it for pushing like this:
variables:
GIT_STRATEGY: clone
build-job:
stage: build
tags:
- data_dict
script:
- echo "Hello!"
- ls
- python3 -V
- pip3 list
test-job1:
stage: test
tags:
- data_dict
script:
- python3 Experiments/ConfAutomation.py
test-job2:
stage: test
tags:
- data_dict
script:
- git show-ref
- git remote -v
- echo "$RUNNER_ACCESS_TOKEN"
- echo "Print runner branch name"
- git config user.email "user#mail.com"
- git config user.name "name"
# - git remote set-url --push origin https://gitlab-ci:"$RUNNER_ACCESS_TOKEN"#gitlab/dir.git
- git add .
- git commit --allow-empty -m "Files from runner to branch"
- git push origin https://gitlab-ci:"$RUNNER_ACCESS_TOKEN"#gitlab/dir.git <branch-name>
deploy-prod:
stage: deploy
tags:
- data_dict
script:
- echo "This job deploys something."
I tried some stuff but can't seem to get it to work. Maybe you had similiar problem or you have some ideas ? Any help would be greatly appreciated.

Try using {$RUNNER_ACCESS_TOKEN} without double quotes.

Related

Running gradle release on gitlab pipeline triggers 3 more build jobs

This is my gitlab pipeline:
image: docker:latest
services:
- docker:dind
variables:
DOCKER_DRIVER: overlay
SPRING_PROFILES_ACTIVE: gitlab-ci
before_script:
- git remote set-url origin https://gitlab-ci-token:$CICD_GIT_TOKEN#$CI_SERVER_HOST/$CI_PROJECT_PATH.git
- git config --global user.email 'me#email.com'
- git config --global user.name 'Gitlab (Me)'
- git checkout -b $CI_COMMIT_BRANCH
stages:
- build
- package
- deploy
gradle-build:
image: gradle:alpine
stage: build
script:
- PROJECT_VERSION=`./gradlew -quiet printVersion`
- ./gradlew release
artifacts:
paths:
- build/libs/*.jar
docker-build:
stage: package
script:
- docker build -t registry.gitlab.com/me/myproject:$PROJECT_VERSION .
- docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN registry.gitlab.com
- docker push registry.gitlab.com/me/myproject:PROJECT_VERSION
Not sure why the 3 builds get triggered (maybe because the plugin does a commit) and not sure how to fix this.
Any help would be appreciated.
Thanks.
After taking a look here:
https://docs.gitlab.com/ee/ci/yaml/
I added the following section to the pipeline:
workflow:
rules:
- if: $CI_COMMIT_AUTHOR =~ /^Gitlab Pipeline/
when: never
- when: always
Moved the stuff in "before_script" under script, and changed the git username:
script:
- git remote set-url origin https://gitlab-ci-token:$CICD_GIT_TOKEN#$CI_SERVER_HOST/$CI_PROJECT_PATH.git
- git config --global user.email 'me#email.com'
- git config --global user.name 'Gitlab Pipeline'
- git checkout -b $CI_COMMIT_BRANCH
- ./gradlew release
Now the commits with the name "Gitlab Pipeline" don't trigger a build anymore.
The extra builds were triggered because the release plugin also does a pre-tag commit, followed by a new version commit at the end. This would trigger 2 builds, which in turn would each trigger 2 more and so on.
Now the pipeline behaves as expected.

Automatic code merging using gitlab-ci.yml pipeline

My goal is to create Gitlab yml file in order to automatically merge code from release to master branch which will be run on a schedule (once a week)
Here is my .gitlab-ci.yml:
image: "python:3.8"
stages:
- build
- test
- release
Build:
stage: build
script:
- <some code here>
Test:
stage: test
script:
- <some code here>
job:on-schedule:
stage: release
only:
- schedules
script:
- git config --global user.email myEmail#gmail.com
- git config --global user.name "myUserName"
- git checkout release
- git checkout master
- git merge release
But I'm getting the following error:
error: pathspec 'release' did not match any file(s) known to git
Cleaning up file based variables
00:00
ERROR: Job failed: exit code 1
. I'd appreciate it if you have any ideas on how to fix that.
By default, gitlab-ci does a shallow clone, and so the error you're seeing of:
error: pathspec 'release' did not match any file(s) known to git
is accurate because the release branch does not exist locally on the runner.
To resolve this, you can just update the lines to:
- git checkout origin/release
- git checkout origin/master
which will pull the latest branches from the repository directly instead.

How to trigger Gitlab CI pipeline manually, when in normal conditions, it is triggered by webhook with commit Ids?

I have Gitlab CI pipeline which is triggered by bitbucket webhook with current and last commit ids. I also want to re-run pipeline manually whenever the build created Gitlab CI file, triggered by webhook is not working as expected.
I tried RUN-PIPELINE option but shows the error:
The form contains the following error:
No stages/jobs for this pipeline.
Here is the GitLab CI file. Include refers to other project where standard yaml file for the pipeline is kept:
include:
- project: Path/to/project
ref: bb-deployment
file: /bitbucket-deployment.yaml
variables:
TILLER_NAMESPACE: <namespace>
NAMESPACE: testenv
REPO_REF: testenvbranch
LastCommitSHA: <commit sha from webhook>
CurrentCommitSHA: <Current commit she from webhook>
Here is the detailed gitlab-ci file that is provided in other project which has stages:
stages:
- pipeline
- build
variables:
ORG: test
APP_NAME: $CI_PROJECT_NAME
before_script:
- 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
- eval $(ssh-agent -s)
- echo "$SSH_PRIIVATE_KEY2" | tr -d '\r' | ssh-add -
- mkdir -p ~/.ssh
- chmod 700 ~/.ssh
- echo "$SSH_KNOWN_HOSTS" > ~/.ssh/known_hosts
- chmod 644 ~/.ssh/known_hosts
Building CI Script:
stage: pipeline
image: python:3.6
only:
refs:
- master
script:
- |
curl https://github.com/org/scripts/branch/install.sh | bash -s latest
source /usr/local/bin/pipeline-variables.sh
git clone git#bitbucket.org:$ORG/$APP_NAME.git
cd $APP_NAME
git checkout $lastCommit
cp -r env old
git checkout $bitbucketCommit
$CMD_DIFF old env
$CMD_BUILD
$CMD_INSTALL updatedReposList.yaml deletedReposList.yaml /tmp/test $NAMESPACE $REPO_REF $ORG $APP_NAME $lastCommit $bitbucketCommit
cat cicd.yaml
mv cicd.yaml ..
artifacts:
paths:
- cicd.yaml
Deplopying Apps:
stage: build
only:
refs:
- master
trigger:
include:
artifact: cicd.yaml
job: Building CI Script
strategy: depend
In the manual trigger, instead of considering the last and current commit she, it should rebuild the application.
Any help will be appreciated.
Thank you for your comment (below), I see you are using the include directive (https://docs.gitlab.com/ce/ci/yaml/#include) in one .gitlab-ci.yml to include a GitLab CI YAML file from another project.
I can duplicate this error (No stages / jobs for this pipeline) by invoking "run pipeline" on project 1 which is configured to include GitLab CI YAML from project 2 when the project 2 GitLab CI YAML is restricted to the master branch but I'm running the project on another branch.
For example, let's say project 1 is called "stackoverflow-test" and its .gitlab-ci.yml is:
include:
- project: atsaloli/test
file: /.gitlab-ci.yml
ref: mybranch
And project 2 is called "test" (in my own namespace, atsaloli) and its .gitlab-ci.yml is:
my_job:
script: echo hello world
image: alpine
only:
refs:
- master
If I select "Run Pipeline" in the GitLab UI in project 1 on a branch other than "master", I then get the error message "No stages / jobs for this pipeline".
That's because there is no job defined for my non-master branch, and then without any job defined, I don't have any stage defined.
I hope that sheds some light on what's going on with your webhook.

gitlab-ci.yml fatal: unable to access http://gitlab-ci-token

I am new to gitlab CI/CD and I'm struggling to figure this out.
All I want to do is when I push to dev branch I want my react app to be built and the folder ./build to be pushed through SSH to my dev server.
Here is what I did so far, including a screenshot of the error message I get.
This is my gitlab-ci.yml
image: node:latest
cache:
paths:
- node_modules/
build_dev:
stage: build
environment: Development
only:
- dev
script:
- ls
- npm install
- npm run build
artifacts:
paths:
- build/
- ecosystem.config.js
deploy_dev:
stage: deploy
environment: Development
only:
- dev
script:
- rsync -r -a -v -e ssh --delete "./build" root#dev.teledirectasia.com:/var/www/gitlab/${CI_PROJECT_NAME}
- rsync -r -a -v -e ssh --delete "./ecosystem.config.js" root#dev.teledirectasia.com:/var/www/gitlab/${CI_PROJECT_NAME}/
- ssh root#dev.teledirectasia.com "cd /var/www/gitlab/${CI_PROJECT_NAME} && pm2 start ecosystem.config.js"
I don't know why I am getting this output with job failed
This is a DNS problem. Your runner cannot resolve the hostname of the GitLab server - gitlab.teledirectgroup.com. Did you set the GitLab hostname if your local workstation's host file manually, or did you set it up in a DNS server as a 'proper' hostname?
If you set up the hostname in a DNS server then the solution may be as simple as adding the DNS server to /etc/resolv.conf on the runner. However, if you just set the GitLab hostname in your workstation's hosts file then you'll need to set it in the runner's /etc/hosts file, too. It's hard to say what the exact solution is without knowing how you set up the GitLab hostname in the first place.
the solution that’s so applied to GitLab?
Use the git clone by ssh, I don’t have a good goal that’s so I can up to push that’s changes over a submodule from runner Shell by GitLab CI. The pipeline ever fails and prints this error.
ERROR PIPELINE JOB
In the local repo as a project the file config contains that line with the URL, more don’t have login with this about the pipeline.
.git/config
Some help or walkthrough of reference to culminate with that challenge in troubleshooting!
This is my code over the file ".gitlab-ci.yml"
variables:
TEST_VAR: "Update Git Submoudel in all Etecnic projects."
job1:
variables: {}
script:
- echo "$TEST_VAR"
job2:
variables: {}
script:
- echo "OK" >> exito.txt
- git add --all
- git commit -m "Update Submodule"
- git push origin HEAD:master
Versions:
GitLab:
gitlab-ce is already the newest version (15.6.0-ce.0).
Runner:
Version: 15.5.1
Git revision: 7178588d
Git branch: 15-5-stable
GO version: go1.18.7
Built: 2022-11-11T09:45:25+0000
OS/Arch: linux/amd64
Thanks so much for your attention.

Do not trigger build when creating new branch from Gitlab issue

I use Gitlab on gitlab.com and the issue tracker.
Each time I have an issue, I create a new branch on the button inside the issue, but this triggers a new build (pipeline) in CI.
I don't want this because this branch is coming from master and doesn't need to be built.
How can I achieve that ? Is this a gitlab-ci.yml modification or a repository related configuration ?
You can define, in which branches particular steps of your build will run, via only and except parameters to the builds: https://docs.gitlab.com/ee/ci/yaml/#only-and-except-complex
For example, run java build in all branches except issue branches:
java-build:
stage: build
except:
- /^issue-.*$/
script:
- mvn -U -e install
image: maven:3.5-jdk-8
Restrict build only to master and release branches:
java-build:
stage: build
only:
- master
- /^RELEASE-.*$/
script:
- mvn -U -e install
image: maven:3.5-jdk-8
Based on the Commit message we can avoid building for the newly created branch on GitLab CI/CD pipeline
Build Branch(master):
stage: build
only:
refs:
- master
variables:
- $CI_COMMIT_MESSAGE =~ /^\[master\].*$/
script:
- echo "master branch"
# - sleep 60 | echo "hello master"
# when: delayed
# start_in: 3 minutes
interruptible: true
Build Branch(release):
stage: build
only:
refs:
- /^build_.*$/
variables:
- $CI_COMMIT_MESSAGE =~ /^\[build\].*$/
script:
- echo "release branch"
# - sleep 60 | echo "hello release"
# when: delayed
# start_in: 3 minutes
interruptible: true
The branches will trigger when the commit message has start with [master] or [build]

Resources