GitLab CI Pipeline: Pipeline cannot be run - gitlab

I can't find out why the GitLab CI Pipelines for my Repo won't run. I have a .gitlab-ci.yml file and the feature enabled, but the pipeline won't run. Also if I try to trigger the pipeline manually I get the following error back.
Pipeline cannot be run.
Pipeline will not run for the selected trigger. The rules configuration prevented any jobs from being added to the pipeline.
The CI feature is enabled.
Here is my .gitlab-ci.yml file.
stages:
- build
- deploy
npm-run-build:
stage: build
image: node:19
only:
- main
cache:
key: ${CI_COMMIT_REF_SLUG}-build
paths:
- dist/
script:
- cp .env.example .env
- npm ci
- npm run build-only
deploy-dist:
stage: deploy
image: fedora:latest
only:
- main
environment:
name: production
url: https://example.com
needs:
- npm-run-build
cache:
key: ${CI_COMMIT_REF_SLUG}-build
paths:
- dist/
before_script:
- dnf install -y openssh-clients
- mkdir -p ~/.ssh
- echo "$SSH_PRIVATE_KEY" > ~/.ssh/id_rsa
- chmod 600 ~/.ssh/id_rsa
- ssh-keyscan -t rsa example.com > ~/.ssh/known_hosts
script:
# create remote project dir if not available
- ssh thomas#example.com "mkdir -p /home/thomas/example.com"
# upload project files
- scp -prq . thomas#example.com:/home/thomas/example.com
# restart the container
- ssh thomas#example.com "cd /home/thomas/example.com && docker-compose down && docker-compose up -d"
Thank you! 😁

As D Malan pointed out in the comments, I have restricted the runs with only to the main branch. But the branch name is actually master 🤦
So I just changed the rule form main to master and now it is running 👌

Related

Gitlab Pipeline: lftp runs into a timeout

I received a notification a few days ago from my gitlab runner about a failed pipeline.
The Pipeline was working normally and nothing got changed, which makes everything a bit harder to investigate.
The specific command which causes the timeout of 1 hour is the following:
lftp -e "set ftp:ssl-allow no; mirror -R dist/ ./; quit;" -u $USER_TEST,$PASSWORD_TEST $HOST_TEST
This was working fine. I tried to troubleshoot this problem. There are a plenty of reasons out there why the timeout happens. However none of it was solving my problem.
A short brief of what the git pipeline does:
building an angular app (the dist folder's size is about 750kB)
deploying it on a server using ftp credentials
I manually went through the pipeline steps, hoping to replicate the bug, but it was working fine.
.gitlab-ci.yml:
image: node:14.15.3-alpine
cache:
paths:
- node_modules/
stages:
- build
- deploy
#DEV Stage
build_stage_dev:
stage: build
only:
refs:
- develop
cache:
paths:
- dist/
script:
- npm install --legacy-peer-deps
- npm install -g #angular/cli#11.0.5
- ng build --build-optimizer
deploy_stage_dev:
stage: deploy
environment: develop
only:
refs:
- develop
script:
- apk update && apk add openssh-client && apk add sshpass
- export SSHPASS=$PASSWORD_DEV
#command to remove all files first
#- sshpass -e ssh -o stricthostkeychecking=no $USER_DEV#$HOST_DEV rm -r /var/www/app/*
- sshpass -e scp -o stricthostkeychecking=no -r dist/* $USER_DEV#$HOST_DEV:/var/www/app
#TEST Stage
build_stage_test:
stage: build
only:
refs:
- test
cache:
paths:
- dist/
script:
- npm install --legacy-peer-deps
- npm install -g #angular/cli#11.0.5
- ng build --build-optimizer
deploy_stage_test:
stage: deploy
environment: test
only:
refs:
- test
script:
- apk update && apk add lftp
- lftp -e "set ftp:ssl-allow no; mirror -R dist/ ./; quit;" -u $USER_TEST,$PASSWORD_TEST $HOST_TEST
The DEV Stage (deploying with the ssh) is working fine. Only the test stage throws an error after 1 hour.
This is the error I receive on Gitlab: ERROR: Job failed: execution took longer than 1h0m0s seconds
Maybe any of you have experienced the same? or did "lftp" got an update and I am running into an enndless job?
Also I checked whether files are getting updated at all, the answer is no. I thought it could be an issue with "quit". But appearently nothing is getting transferred at all.

I get no such file or directory when moving dist folder contents from gitlab to aws server instance

I am new to Gitlab CI/CD and trying to fix this all day long but nothing works. I am trying to move the dist folder generated by gitlab runner after build stage to aws ec2-instance folder location. I am currently implementing CI/CD pipeline using Gitlab and this is how my .gitlab-ci.yml looks like:
# Node Image for docker on which code will execute
image: node:latest
# This is the stages / task to perfom in jobs
stages:
- build
- deploy
# caching for reuse
cache:
key: "$CI_COMMIT_REF_SLUG"
paths:
- node_modules/
# This command is run before the execution of stages
before_script:
- npm install
# Job One for making build
build_testing_branch:
stage: build
script:
- node --max_old_space_size=4096 --openssl-legacy-provider ./node_modules/#angular/cli/bin/ng build --configuration=dev-build --build-optimizer
only: ['testing']
# Job Two for deploy build to server
deploy_testing_branch:
stage: deploy
before_script:
- 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
- eval $(ssh-agent -s)
# - echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add -
- mkdir -p ~/.ssh
- chmod 700 ~/.ssh
- echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config
# - apt-get update -y
# - apt-get -y install rsync
artifacts:
paths:
- dist/
script:
- ssh-add <(echo "$SSH_PRIVATE_KEY")
- ssh -p22 ubuntu#$SERVER_IP "rm -r /usr/share/nginx/user-host/ui-user-host/dist/; mkdir /usr/share/nginx/user-host/ui-user-host/dist/"
- scp -P22 -r $CI_PROJECT_DIR/dist/ ubuntu#$SERVER_IP:/usr/share/nginx/user-host/ui-user-host/dist/
only: ['testing']
The build process works just fine with success confirmation, but the deployment stage fails because I get:
$scp -P22 -r $CI_PROJECT_DIR/dist/ ubuntu#$SERVER_IP:/usr/share/nginx/user-host/ui-user-host/dist/
/builds/user-live/ui-user-host/dist: No such file or directory
Cleaning up project directory and file based variables
So, I dont understand why its not able to locate dist folder in above location. If i correctly understand this should be available on the gitlab runner's filesystem. Is it because scp command is not right?
EDIT:
I also tried with
- scp -P22 -r dist/ ubuntu#$SERVER_IP:/usr/share/nginx/user-host/ui-user-host/dist/
and
- scp -P22 -r dist/* ubuntu#$SERVER_IP:/usr/share/nginx/user-host/ui-user-host/dist/
but no luck!
You are building your dist folder in build_testing_branch job and trying to access it in deploy_testing_branch for this to work you have to give the dist folder as artifact in build_testing_branch job (since dist is created there) and not in deploy_testing_branch.

Sonarcloud shows 0% coverage on new code, and also shows 0% coverage on master branch with gitlab ci

I am using GitLab ci to run SonarCloud code analysis on the code.
here is my gitlab-ci.yaml
stages:
- test
before_script:
- mkdir -p ~/.ssh &&
cp $gitlab_private_key ~/.ssh/id_ed25519 &&
chmod 600 ~/.ssh/id_ed25519 &&
touch ~/.ssh/known_hosts &&
ssh-keyscan gitlab.com >> ~/.ssh/``known_hosts
variables:
SONAR_USER_HOME: "${CI_PROJECT_DIR}/.sonar" # Defines the location of the analysis task cache
GIT_DEPTH: "0" # Tells git to fetch all the branches of the project, required by the analysis task
GITLAB_PROJECT_ID: ${CI_PROJECT_ID} # needed to be exported to the project's environments
FLASK_APP: manage.py
sonarcloud-check:
image:
name: sonarsource/sonar-scanner-cli:latest
entrypoint: [""]
cache:
key: "${CI_JOB_NAME}"
paths:
- .sonar/cache
script:
- sonar-scanner
only:
- merge_requests
- master
test-merge-request-changes:
stage: test
only:
- merge_requests
image:
name: docker:19.03.13-git
services:
- name: docker:19.03.0-dind
entrypoint: ["env", "-u", "DOCKER_HOST"]
command: ["dockerd-entrypoint.sh"]
variables:
DOCKER_HOST: tcp://localhost:2375
DOCKER_TLS_CERTDIR: ""
DOCKER_DRIVER: overlay2
ENV: test
CI_DEBUG_TRACE: "true"
before_script:
- echo $CI_BUILD_TOKEN | docker login -u gitlab-ci-token --password-stdin ${CI_REGISTRY}
script:
- echo "Running Tests..."
- cp ${group_shared_vars} ${CI_PROJECT_DIR}/.env
- docker build . -f Dockerfile-testing -t test_merge_req --build-arg GITLAB_PROJECT_ID=${GITLAB_PROJECT_ID}
- docker run --cidfile="my-package.txt" test_merge_req:latest
after_script:
- touch text2.txt
- docker cp $(cat my-package.txt):/app/tests/coverage/coverage.xml coverage.xml
- docker cp $(cat my-package.txt):/app/tests/coverage/junit.xml junit.xml
timeout: 2h
artifacts:
when: always
reports:
cobertura:
- coverage.xml
junit:
- junit.xml
coverage: '/TOTAL.*\s+(\d+%)$/'
And here is my sonar-project.properties
sonar.projectKey=my_app-key
sonar.organization=my_org
sonar.sources=lib
sonar.tests=tests
sonar.exclusions=tests
sonar.language=python
sonar.python.version=3.8
I want to get the report that is generated in the container analyzed by sonarcloud on each merge request.
Also, when a code is pushed to the master branch, I want to get the coverage percent on sonarcloud of the project to be updated but it just shows 0%.
Is there any way that after the merge requests are run, we get the sonarcloud analysis on the report of the docker container?
And also getting the master branch coverage updated without having to commit the coverage.xml to the repo?
After digging into how Gitlab CI stages and jobs work and also the insight that this thread brought, I have tweaked the above GitLab ci so that it :
First runs the tests
Then uploads the coverage outputs into the path specified for the artifacts
Then in the next stage, it pulls those artifacts
Replaces the path that was replaced by pytest-cov in the <source>...</source> with a sed command. (This step is due to the path that was placed into the source element is relevant to the docker container and then after downloading the coverage report not working inside the GitLab ci container)
Running the sonar-scanner on the coverage report.
GitLab ci:
stages:
- build_test
- analyze_test
before_script:
- mkdir -p ~/.ssh &&
cp $gitlab_private_key ~/.ssh/id_ed25519 &&
chmod 600 ~/.ssh/id_ed25519 &&
touch ~/.ssh/known_hosts &&
ssh-keyscan gitlab.com >> ~/.ssh/``known_hosts
variables:
SONAR_USER_HOME: "${CI_PROJECT_DIR}/.sonar" # Defines the location of the analysis task cache
GIT_DEPTH: "0" # Tells git to fetch all the branches of the project, required by the analysis task
GITLAB_PROJECT_ID: ${CI_PROJECT_ID} # needed to be exported to the projects environments
include:
- template: Code-Quality.gitlab-ci.yml
test-merge-request:
stage: build_test
only:
- merge_requests
- master
image:
name: docker:19.03.13-git
cache:
key: "${CI_JOB_NAME}"
paths:
- build
services:
- name: docker:19.03.0-dind
entrypoint: ["env", "-u", "DOCKER_HOST"]
command: ["dockerd-entrypoint.sh"]
variables:
DOCKER_HOST: tcp://localhost:2375
DOCKER_TLS_CERTDIR: ""
DOCKER_DRIVER: overlay2
ENV: test
CI_DEBUG_TRACE: "true"
before_script:
- echo $CI_BUILD_TOKEN | docker login -u gitlab-ci-token --password-stdin ${CI_REGISTRY}
script:
- echo "Running Tests..."
- cp ${group_shared_vars} ${CI_PROJECT_DIR}/.env
- docker build . -f Dockerfile-testing -t test_merge_req --build-arg GITLAB_PROJECT_ID=${GITLAB_PROJECT_ID}
- docker run --cidfile="my-package.txt" test_merge_req:latest
after_script:
- docker cp $(cat my-package.txt):/app/tests/coverage/coverage.xml build/coverage.xml
- docker cp $(cat my-package.txt):/app/tests/coverage/junit.xml build/junit.xml
timeout: 2h
artifacts:
when: always
paths:
- build
reports:
cobertura:
- build/coverage.xml
junit:
- build/junit.xml
expire_in: 30 min
coverage: '/TOTAL.*\s+(\d+%)$/'
sonarcloud-check:
stage: analyze_test
image:
name: sonarsource/sonar-scanner-cli:latest
entrypoint: [""]
cache:
key: "${CI_JOB_NAME}"
paths:
- .sonar/cache
script:
- echo "Logging coverage..."
- sed -i "s|<source>\/app\/my_app<\/source>|<source>$CI_PROJECT_DIR\/my_app<\/source>|g" ./build/coverage.xml
- sonar-scanner
only:
- merge_requests
- master
dependencies:
- test-merge-request-changes
The takeaway from my issue was that Gitlab doesn't share the artifacts between the jobs that are placed in one GitLab ci stage, but rather shares them between different stages.
So, I just created two separate stages, one for building the tests (build_test) and one for analyzing the tests (analyze_test). Then, made the sonnar-cloud job dependent on the test-merge-request-changes job. This way, we make sure that first the tests runs in the test-merge-request-changes and then the uploaded artifacts are utilized inside the sonar-cloud stage.

edit and execute pipeline *.yml template file from command line [duplicate]

If a GitLab project is configured on GitLab CI, is there a way to run the build locally?
I don't want to turn my laptop into a build "runner", I just want to take advantage of Docker and .gitlab-ci.yml to run tests locally (i.e. it's all pre-configured). Another advantage of that is that I'm sure that I'm using the same environment locally and on CI.
Here is an example of how to run Travis builds locally using Docker, I'm looking for something similar with GitLab.
Since a few months ago this is possible using gitlab-runner:
gitlab-runner exec docker my-job-name
Note that you need both docker and gitlab-runner installed on your computer to get this working.
You also need the image key defined in your .gitlab-ci.yml file. Otherwise won't work.
Here's the line I currently use for testing locally using gitlab-runner:
gitlab-runner exec docker test --docker-volumes "/home/elboletaire/.ssh/id_rsa:/root/.ssh/id_rsa:ro"
Note: You can avoid adding a --docker-volumes with your key setting it by default in /etc/gitlab-runner/config.toml. See the official documentation for more details. Also, use gitlab-runner exec docker --help to see all docker-based runner options (like variables, volumes, networks, etc.).
Due to the confusion in the comments, I paste here the gitlab-runner --help result, so you can see that gitlab-runner can make builds locally:
gitlab-runner --help
NAME:
gitlab-runner - a GitLab Runner
USAGE:
gitlab-runner [global options] command [command options] [arguments...]
VERSION:
1.1.0~beta.135.g24365ee (24365ee)
AUTHOR(S):
Kamil Trzciński <ayufan#ayufan.eu>
COMMANDS:
exec execute a build locally
[...]
GLOBAL OPTIONS:
--debug debug mode [$DEBUG]
[...]
As you can see, the exec command is to execute a build locally.
Even though there was an issue to deprecate the current gitlab-runner exec behavior, it ended up being reconsidered and a new version with greater features will replace the current exec functionality.
Note that this process is to use your own machine to run the tests using docker containers. This is not to define custom runners. To do so, just go to your repo's CI/CD settings and read the documentation there. If you wanna ensure your runner is executed instead of one from gitlab.com, add a custom and unique tag to your runner, ensure it only runs tagged jobs and tag all the jobs you want your runner to be responsible of.
I use this docker-based approach:
Edit: 2022-10
docker run --entrypoint bash --rm -w $PWD -v $PWD:$PWD -v /var/run/docker.sock:/var/run/docker.sock gitlab/gitlab-runner:latest -c 'git config --global --add safe.directory "*";gitlab-runner exec docker test'
For all git versions > 2.35.2. You must add safe.directory within the container to avoid fatal: detected dubious ownership in repository at.... This also true for patched git versions < 2.35.2. The old command will not work anymore.
Details
0. Create a git repo to test this answer
mkdir my-git-project
cd my-git-project
git init
git commit --allow-empty -m"Initialize repo to showcase gitlab-runner locally."
1. Go to your git directory
cd my-git-project
2. Create a .gitlab-ci.yml
Example .gitlab-ci.yml
image: alpine
test:
script:
- echo "Hello Gitlab-Runner"
3. Create a docker container with your project dir mounted
docker run -d \
--name gitlab-runner \
--restart always \
-v $PWD:$PWD \
-v /var/run/docker.sock:/var/run/docker.sock \
gitlab/gitlab-runner:latest
(-d) run container in background and print container ID
(--restart always) or not?
(-v $PWD:$PWD) Mount current directory into the current directory of the container - Note: On Windows you could bind your dir to a fixed location, e.g. -v ${PWD}:/opt/myapp. Also $PWD will only work at powershell not at cmd
(-v /var/run/docker.sock:/var/run/docker.sock) This gives the container access to the docker socket of the host so it can start "sibling containers" (e.g. Alpine).
(gitlab/gitlab-runner:latest) Just the latest available image from dockerhub.
4. Execute with
Avoid fatal: detected dubious ownership in repository at... More info
docker exec -it -w $PWD gitlab-runner git config --global --add safe.directory "*"
Actual execution
docker exec -it -w $PWD gitlab-runner gitlab-runner exec docker test
# ^ ^ ^ ^ ^ ^
# | | | | | |
# (a) (b) (c) (d) (e) (f)
(a) Working dir within the container. Note: On Windows you could use a fixed location, e.g. /opt/myapp.
(b) Name of the docker container
(c) Execute the command "gitlab-runner" within the docker container
(d)(e)(f) run gitlab-runner with "docker executer" and run a job named "test"
5. Prints
...
Executing "step_script" stage of the job script
$ echo "Hello Gitlab-Runner"
Hello Gitlab-Runner
Job succeeded
...
Note: The runner will only work on the commited state of your code base. Uncommited changes will be ignored. Exception: The .gitlab-ci.yml itself does not have be commited to be taken into account.
Note: There are some limitations running locally. Have a look at limitations of gitlab runner locally.
I'm currently working on making a gitlab runner that works locally.
Still in the early phases, but eventually it will become very relevant.
It doesn't seem like gitlab want/have time to make this, so here you go.
https://github.com/firecow/gitlab-runner-local
If you are running Gitlab using the docker image there: https://hub.docker.com/r/gitlab/gitlab-ce, it's possible to run pipelines by exposing the local docker.sock with a volume option: -v /var/run/docker.sock:/var/run/docker.sock. Adding this option to the Gitlab container will allow your workers to access to the docker instance on the host.
The GitLab runner appears to not work on Windows yet and there is an open issue to resolve this.
So, in the meantime I am moving my script code out to a bash script, which I can easily map to a docker container running locally and execute.
In this case I want to build a docker container in my job, so I create a script 'build':
#!/bin/bash
docker build --pull -t myimage:myversion .
in my .gitlab-ci.yaml I execute the script:
image: docker:latest
services:
- docker:dind
before_script:
- apk add bash
build:
stage: build
script:
- chmod 755 build
- build
To run the script locally using powershell I can start the required image and map the volume with the source files:
$containerId = docker run --privileged -d -v ${PWD}:/src docker:dind
install bash if not present:
docker exec $containerId apk add bash
Set permissions on the bash script:
docker exec -it $containerId chmod 755 /src/build
Execute the script:
docker exec -it --workdir /src $containerId bash -c 'build'
Then stop the container:
docker stop $containerId
And finally clean up the container:
docker container rm $containerId
Another approach is to have a local build tool that is installed on your pc and your server at the same time.
So basically, your .gitlab-ci.yml will basically call your preferred build tool.
Here an example .gitlab-ci.yml that i use with nuke.build:
stages:
- build
- test
- pack
variables:
TERM: "xterm" # Use Unix ASCII color codes on Nuke
before_script:
- CHCP 65001 # Set correct code page to avoid charset issues
.job_template: &job_definition
except:
- tags
build:
<<: *job_definition
stage: build
script:
- "./build.ps1"
test:
<<: *job_definition
stage: test
script:
- "./build.ps1 test"
variables:
GIT_CHECKOUT: "false"
pack:
<<: *job_definition
stage: pack
script:
- "./build.ps1 pack"
variables:
GIT_CHECKOUT: "false"
only:
- master
artifacts:
paths:
- output/
And in nuke.build i've defined 3 targets named like the 3 stages (build, test, pack)
In this way you have a reproducible setup (all other things are configured with your build tool) and you can test directly the different targets of your build tool.
(i can call .\build.ps1 , .\build.ps1 test and .\build.ps1 pack when i want)
I am on Windows using VSCode with WSL
I didn't want to register my work PC as a runner so instead I'm running my yaml stages locally to test them out before I upload them
$ sudo apt-get install gitlab-runner
$ gitlab-runner exec shell build
yaml
image: node:10.19.0 # https://hub.docker.com/_/node/
# image: node:latest
cache:
# untracked: true
key: project-name
# key: ${CI_COMMIT_REF_SLUG} # per branch
# key:
# files:
# - package-lock.json # only update cache when this file changes (not working) #jkr
paths:
- .npm/
- node_modules
- build
stages:
- prepare # prepares builds, makes build needed for testing
- test # uses test:build specifically #jkr
- build
- deploy
# before_install:
before_script:
- npm ci --cache .npm --prefer-offline
prepare:
stage: prepare
needs: []
script:
- npm install
test:
stage: test
needs: [prepare]
except:
- schedules
tags:
- linux
script:
- npm run build:dev
- npm run test:cicd-deps
- npm run test:cicd # runs puppeteer tests #jkr
artifacts:
reports:
junit: junit.xml
paths:
- coverage/
build-staging:
stage: build
needs: [prepare]
only:
- schedules
before_script:
- apt-get update && apt-get install -y zip
script:
- npm run build:stage
- zip -r build.zip build
# cache:
# paths:
# - build
# <<: *global_cache
# policy: push
artifacts:
paths:
- build.zip
deploy-dev:
stage: deploy
needs: [build-staging]
tags: [linux]
only:
- schedules
# # - branches#gitlab-org/gitlab
before_script:
- apt-get update && apt-get install -y lftp
script:
# temporarily using 'verify-certificate no'
# for more on verify-certificate #jkr: https://www.versatilewebsolutions.com/blog/2014/04/lftp-ftps-and-certificate-verification.html
# variables do not work with 'single quotes' unless they are "'surrounded by doubles'"
- lftp -e "set ssl:verify-certificate no; open mediajackagency.com; user $LFTP_USERNAME $LFTP_PASSWORD; mirror --reverse --verbose build/ /var/www/domains/dev/clients/client/project/build/; bye"
# environment:
# name: staging
# url: http://dev.mediajackagency.com/clients/client/build
# # url: https://stg2.client.co
when: manual
allow_failure: true
build-production:
stage: build
needs: [prepare]
only:
- schedules
before_script:
- apt-get update && apt-get install -y zip
script:
- npm run build
- zip -r build.zip build
# cache:
# paths:
# - build
# <<: *global_cache
# policy: push
artifacts:
paths:
- build.zip
deploy-client:
stage: deploy
needs: [build-production]
tags: [linux]
only:
- schedules
# - master
before_script:
- apt-get update && apt-get install -y lftp
script:
- sh deploy-prod
environment:
name: production
url: http://www.client.co
when: manual
allow_failure: true
The idea is to keep check commands outside of .gitlab-ci.yml. I use Makefile to run something like make check and my .gitlab-ci.yml runs the same make commands that I use locally to check various things before committing.
This way you'll have one place with all/most of your commands (Makefile) and .gitlab-ci.yml will have only CI-related stuff.
I have written a tool to run all GitLab-CI job locally without have to commit or push, simply with the command ci-toolbox my_job_name.
The URL of the project : https://gitlab.com/mbedsys/citbx4gitlab
Years ago I build this simple solution with Makefile and docker-compose to run the gitlab runner in docker, you can use it to execute jobs locally as well and should work on all systems where docker works:
https://gitlab.com/1oglop1/gitlab-runner-docker
There are few things to change in the docker-compose.override.yaml
version: "3"
services:
runner:
working_dir: <your project dir>
environment:
- REGISTRATION_TOKEN=<token if you want to register>
volumes:
- "<your project dir>:<your project dir>"
Then inside your project you can execute it the same way as mentioned in other answers:
docker exec -it -w $PWD runner gitlab-runner exec <commands>..
I recommend using gitlab-ci-local
https://github.com/firecow/gitlab-ci-local
It's able to run specific jobs as well.
It's a very cool project and I have used it to run simple pipelines on my laptop.

How to trigger Gitlab CI pipeline manually, when in normal conditions, it is triggered by webhook with commit Ids?

I have Gitlab CI pipeline which is triggered by bitbucket webhook with current and last commit ids. I also want to re-run pipeline manually whenever the build created Gitlab CI file, triggered by webhook is not working as expected.
I tried RUN-PIPELINE option but shows the error:
The form contains the following error:
No stages/jobs for this pipeline.
Here is the GitLab CI file. Include refers to other project where standard yaml file for the pipeline is kept:
include:
- project: Path/to/project
ref: bb-deployment
file: /bitbucket-deployment.yaml
variables:
TILLER_NAMESPACE: <namespace>
NAMESPACE: testenv
REPO_REF: testenvbranch
LastCommitSHA: <commit sha from webhook>
CurrentCommitSHA: <Current commit she from webhook>
Here is the detailed gitlab-ci file that is provided in other project which has stages:
stages:
- pipeline
- build
variables:
ORG: test
APP_NAME: $CI_PROJECT_NAME
before_script:
- 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
- eval $(ssh-agent -s)
- echo "$SSH_PRIIVATE_KEY2" | tr -d '\r' | ssh-add -
- mkdir -p ~/.ssh
- chmod 700 ~/.ssh
- echo "$SSH_KNOWN_HOSTS" > ~/.ssh/known_hosts
- chmod 644 ~/.ssh/known_hosts
Building CI Script:
stage: pipeline
image: python:3.6
only:
refs:
- master
script:
- |
curl https://github.com/org/scripts/branch/install.sh | bash -s latest
source /usr/local/bin/pipeline-variables.sh
git clone git#bitbucket.org:$ORG/$APP_NAME.git
cd $APP_NAME
git checkout $lastCommit
cp -r env old
git checkout $bitbucketCommit
$CMD_DIFF old env
$CMD_BUILD
$CMD_INSTALL updatedReposList.yaml deletedReposList.yaml /tmp/test $NAMESPACE $REPO_REF $ORG $APP_NAME $lastCommit $bitbucketCommit
cat cicd.yaml
mv cicd.yaml ..
artifacts:
paths:
- cicd.yaml
Deplopying Apps:
stage: build
only:
refs:
- master
trigger:
include:
artifact: cicd.yaml
job: Building CI Script
strategy: depend
In the manual trigger, instead of considering the last and current commit she, it should rebuild the application.
Any help will be appreciated.
Thank you for your comment (below), I see you are using the include directive (https://docs.gitlab.com/ce/ci/yaml/#include) in one .gitlab-ci.yml to include a GitLab CI YAML file from another project.
I can duplicate this error (No stages / jobs for this pipeline) by invoking "run pipeline" on project 1 which is configured to include GitLab CI YAML from project 2 when the project 2 GitLab CI YAML is restricted to the master branch but I'm running the project on another branch.
For example, let's say project 1 is called "stackoverflow-test" and its .gitlab-ci.yml is:
include:
- project: atsaloli/test
file: /.gitlab-ci.yml
ref: mybranch
And project 2 is called "test" (in my own namespace, atsaloli) and its .gitlab-ci.yml is:
my_job:
script: echo hello world
image: alpine
only:
refs:
- master
If I select "Run Pipeline" in the GitLab UI in project 1 on a branch other than "master", I then get the error message "No stages / jobs for this pipeline".
That's because there is no job defined for my non-master branch, and then without any job defined, I don't have any stage defined.
I hope that sheds some light on what's going on with your webhook.

Resources