Gitlab Yamls does not works properly Job needs - gitlab

I'm facing some issues and would be glad if someone can help me.
My main goal here it's being able to save at the same pipeline, more than one job. The error it’s because the jobs does not exist for the specific branch, since I have 6 branches I don't want to modified one by one. I would like to have only one Yaml version.
When I've tried to save, GitLab show me the following errors.
Found errors in your .gitlab-ci.yml:
jobs:deploy_dc_manual:needs:need job should be a string
You can test your .gitlab-ci.yml in CI Lint.
This, it is a piece of the code. Example if I am in the branch feature the job needs it´s "feature_package_build" If I am at integration branch will be expected "int_package_build".
feature_package_build:
extends: .build
only:
- /^feature\/.*/
script:
# GitLab API query
- LAST=$(curl -s "sensitive data" | jq '.[0] | .sha' | sed '1q;d' | sed 's:^.\(.*\).$:\1:')
- >
if [ "$OLDER_COMMIT" == "none" ]; then
node_modules/sfdx-cli/bin/run sfpowerkit:project:diff -d package -r ${LAST} -x --loglevel debug
elif [ "$OLDER_COMMIT" != "none" ]; then
node_modules/sfdx-cli/bin/run sfpowerkit:project:diff -d package -r $OLDER_COMMIT -x --loglevel debug
fi
int_package_build:
extends: .build
only:
- integration
script:
# GitLab API query
- LAST=$(curl -s "sensitive data" | jq '.[0] | .sha' | sed '1q;d' | sed 's:^.\(.*\).$:\1:')
- >
if [ "$OLDER_COMMIT" == "none" ]; then
node_modules/sfdx-cli/bin/run sfpowerkit:project:diff -d package -r ${LAST} -x --loglevel debug
elif [ "$OLDER_COMMIT" != "none" ]; then
node_modules/sfdx-cli/bin/run sfpowerkit:project:diff -d package -r $OLDER_COMMIT -x --loglevel debug
fi
uat_package_build:
extends: .build
only:
- uat
script:
# GitLab API query
- LAST=$(curl -s "sensitive data" | jq '.[0] | .sha' | sed '1q;d' | sed 's:^.\(.*\).$:\1:')
deploy_DC_Manual:
extends:
- .deployDC_Manual
needs:
- job:
if [uat_package_build]; then
fi
if [feature_package_build]; then
fi
only:
- /^feature\/.*/
- integration
- uat
- release
- master
script:
- nomDeployedDC=${NONDEPLOYEDDC}
- >
if [ -f package/destructiveChanges.xml ] && [ "$VALIDATE" == "no" ]; then

I assume what you are looking for is the optional-flag for the needs keyword like
deploy_DC_Manual:
extends:
- .deployDC_Manual
needs:
- job: uat_package_build
optional: true
- job: feature_package_build
optional: true
As this makes the needs conditional - but please be aware, that this also means if you mess up your rules, and none of those two is available, your job might be still executed.

Related

Use multiline string in Gitlab release-cli description

I want to create a release in the gitlab-cli with a multiline description but when add one by yaml specs I get the following error:
Executing "step_release" stage of the job script
/bin/sh: eval: line 150: syntax error: unexpected newline
$ release-cli create --name "Release $CI_COMMIT_TAG" --description "Release created using the release-cli. # collapsed multi-line command
Cleaning up project directory and file based variables
ERROR: Job failed: exit code 2
Job:
create release:
...
release:
tag_name: $CI_COMMIT_TAG
name: 'Release $CI_COMMIT_TAG'
description: |-
Release created using the release-cli.
Maven dependency
```xml
<dependency>
<groupId>group</groupId>
<artifactId>artifact-id</artifactId>
<version>$CI_COMMIT_TAG</version>
</dependency>
```
(also tried with > and |)
I tried it manually with \n but that is literally added in the release as
Release created using the release-cli.\n\n ...
How do I create a release via the release-cli with a multiline (markdown) description?
I found a dirty unreadable workaround but it works.
(Note that multiline echo doesn't work either)
create release:
...
image: registry.gitlab.com/gitlab-org/release-cli:latest
rules:
script:
- echo "Running the release job for version '$CI_COMMIT_TAG'."
- set -e
- |
echo "Release created using the release-cli." > release_notes.md
echo "" >> release_notes.md
echo "Maven dependency:" >> release_notes.md
echo "\`\`\`xml" >> release_notes.md
echo "<dependency>" >> release_notes.md
echo " <groupId>group</groupId>" >> release_notes.md
echo " <artifactId>artifact-id</artifactId>" >> release_notes.md
echo " <version>$CI_COMMIT_TAG</version>" >> release_notes.md
echo "</dependency>" >> release_notes.md
echo "\`\`\`" >> release_notes.md
release:
tag_name: $CI_COMMIT_TAG
name: 'Release $CI_COMMIT_TAG'
description: release_notes.md
Still searching for a better solution.

If condition is failing in the GitLab CI

following is my Gitlab CI code:-
stages:
- check
variables:
JIRA_HEADER: "Accept: application/json"
jira:
stage: check
before_script:
#- apk add jq curl
- apk add --no-cache bash jq curl
image: python:3.7.4-alpine3.9
script:
- export MERGE_REQUEST_JIRA_ID=$(echo ${CI_MERGE_REQUEST_TITLE} | sed -r "s/^([A-Za-z][A-Za-z0-9]+-[0-9]+).*/\1/")
- echo $CI_MERGE_REQUEST_TITLE
- export JIRA_DETAIL=$(curl -u ${JIRA_USERNAME}:${JIRA_PASSWORD} -H "${JIRA_HEADER}" -X GET https://${JIRA_SERVER}/rest/api/2/issue/${MERGE_REQUEST_JIRA_ID}?fields=status)
- echo $JIRA_DETAIL
# extract the JIRA key id, this also validates JIRA issue referenced is valid
- export JIRA_KEY_ID=$(echo ${JIRA_DETAIL} | jq -e '.key')
- echo $JIRA_KEY_ID
# extract the JIRA status
- export JIRA_STATUS=$(echo ${JIRA_DETAIL} | jq '.fields.status.name')
- echo $JIRA_STATUS
- |
if [[ "$JIRA_STATUS" == "^(Done|Completed|Closed)$" ]]
then
echo "Invalid JIRA (Done/Completed/Closed) found!"
exit 1
else echo "Valid JIRA Id found!"
fi
only:
- merge_requests
I'm trying to validate the JIRA status by calling its API after retrieving Jira id from title of Merge Request. There is a problem in the If condition below if [[ "$JIRA_STATUS" == "^(Done|Completed|Closed)$" ]] as it is not validating it properly. Every time, the else condition is getting executed and printing the message as Valid JIRA Id found!
I would really appreciate if someone can help me to fix this minor issue. I want to gracefully exit the job with this message in the if block as Invalid JIRA (Done/Completed/Closed) found! whenever the Jira status found to be in any of the given values as Done, Completed or Closed.
I'm finally able to resolve this issue by modifying the code like below:-
- |
if test -z "$(echo ${JIRA_STATUS} | sed -r "s/\"(Done|Completed|Closed)\"//")"
then
echo "Not a valid Jira (Done/Completed/Closed)"; exit 1
else
echo "Valid Jira found!"; echo $?
fi
I had used the test command along with if-else condition in Linux to make it work

Prevent trigger Gitlab-ci job after git tagging

I tried to create templates for other projects to create docker images.
My stages are create tag id, lint our code, and git tag push. ( We push our tag via GitLab API with project access token) I can create a Docker image with our template, but our pipeline tried to run again after our GitLab API tag request and got an error. ( Project not found or access denied) I tried several things to prevent rebuild. ( I used git push -o ci.skip with the personal access token. It seems OK, but we don't want to use a personal access token in our pipelines) Could anyone help with that?
You can see my gitlab-ci template:
---
variables:
http_proxy: ""
https_proxy: ""
no_proxy: ""
stages:
- tag
- lint
- build
- tagRelease
lint:
stage: lint
image:
script:
- hadolint Dockerfile
- |
if ls ./*.sh 1> /dev/null 2>&1 ; then
echo "-------SH SCRIPT FOUND------------"
shellcheck ./*.sh
fi
if ls ./*.bash 1> /dev/null 2>&1 ; then
echo "-------BASH SCRIPT FOUND------------"
shellcheck ./*.bash
fi
tags:
- DOCKER
- AWS
createTag:
stage: tag
image:
name:
script:
- |
LAST_BUILD_ID=$(git tag -l | sort -V | tail -n 1 | cut -c 6,7,8 )
echo "LAST_BUILD_ID=${LAST_BUILD_ID}"
if [ -n "${LAST_BUILD_ID}" ] ; then
BUILD_INDEX=$(( LAST_BUILD_ID + 1 ))
else
BUILD_INDEX=v1.0.0
fi
echo "BUILD_INDEX=${BUILD_INDEX}"
export TAG="v1.0.${BUILD_INDEX}"
echo "TAG=${TAG}"
echo "TAG=${TAG}" > id.env
only:
refs:
- master
artifacts:
reports:
dotenv: id.env
tags:
- DOCKER
- AWS
buildImageDevWithNoPush:
stage: build
image:
name:
entrypoint: [""]
script:
- mkdir -p /kaniko/.docker
- echo "{\"auths\":{\"$CI_REGISTRY\":{\"username\":\"$CI_REGISTRY_USER\",\"password\":\"$CI_REGISTRY_PASSWORD\"}}}" > /kaniko/.docker/config.json
- echo "-----------DOCKER BUILD STARTED. IMAGE WILL NOT BE PUSHED-------------"
- /kaniko/executor --context $CI_PROJECT_DIR --dockerfile $CI_PROJECT_DIR/Dockerfile --no-push --build-arg http_proxy=$http_proxy --build-arg https_proxy=$https_proxy --build-arg no_proxy=$no_proxy
rules:
- if: '$CI_COMMIT_BRANCH =~ /^feature\/.+$/'
tags:
- DOCKER
- AWS
buildImageDevWithPush:
stage: build
image:
name:
entrypoint: [""]
script:
- mkdir -p /kaniko/.docker
- echo "{\"auths\":{\"$CI_REGISTRY\":{\"username\":\"$CI_REGISTRY_USER\",\"password\":\"$CI_REGISTRY_PASSWORD\"}}}" > /kaniko/.docker/config.json
- echo "-----------DOCKER BUILD STARTED. IMAGE WILL NOT BE PUSHED-------------"
- /kaniko/executor --context $CI_PROJECT_DIR --dockerfile $CI_PROJECT_DIR/Dockerfile --destination $CI_REGISTRY_IMAGE:feature-${CI_COMMIT_SHORT_SHA} --build-arg http_proxy=$http_proxy --build-arg https_proxy=$https_proxy --build-arg no_proxy=$no_proxy
rules:
- if: '$CI_COMMIT_BRANCH =~ /^feature\/.+$/'
when: manual
tags:
- DOCKER
- AWS
ProdBuildImage:
stage: build
image:
name:
entrypoint: [""]
script:
- echo ${TAG}
- mkdir -p /kaniko/.docker
- echo "{\"auths\":{\"$CI_REGISTRY\":{\"username\":\"$CI_REGISTRY_USER\",\"password\":\"$CI_REGISTRY_PASSWORD\"}}}" > /kaniko/.docker/config.json
- echo "---------DOCKER BUILD STARTED------------"
- /kaniko/executor --context $CI_PROJECT_DIR --dockerfile $CI_PROJECT_DIR/Dockerfile --destination $CI_REGISTRY_IMAGE:$TAG --build-arg http_proxy=$http_proxy --build-arg https_proxy=$https_proxy --build-arg no_proxy=$no_proxy
only:
refs:
- master
dependencies:
- createTag
tags:
- DOCKER
- AWS
- AWS
.tagRelease:
stage: tagRelease
script:
- curl -o /dev/null -w "%{http_code}" -X POST -H "PRIVATE-TOKEN:$CSC_API_TOKEN" --fail "$CI_API_V4_URL/projects/$CI_PROJECT_ID/repository/tags?tag_name=$TAG&ref=${CI_COMMIT_SHORT_SHA}"
Template Included by other project:
---
include:
- project: ''
ref: 'feature/pipeline_template'
file: 'docker-build-template.yml'
tagRelease:
extends: .tagRelease
tags:
- AWS
- DOCKER
except:
- tags

busybox v1.22.1 multicall binary error on copy command in GitLab

I am trying to copy files as part of GitLab pipeline but I am getting
busybox v1.22.1 multicall binary error on copy command. It was working earlier but suddently showing this error.
Here is my script
script:
- mkdir -p ./input
- git log -m -2 --name-only --diff-filter=d --pretty="format:" >
./input/changes.lst
- |
file="./input/changes.lst"
while IFS= read -r line
do
printf '%s\n' "$line";
if [ -e $line ]
then
`cp -R $line ./input/`;
fi
done < "$file"
only:
- master
artifacts:
when: always
paths:
- input

Gitlab manual steps are skipped by default. How to stop that?

Currently, In my gitlab configuration workflow, i have some manual stage tests. So I can decide if the test pass or fail. Now the manual steps are always skipped by default. Whenever the normal stage steps are building it jump to another normal stage without considering manual steps. Now, How i can make it work. Please help me in this.
stages:
- start_pipeline
- auto_testing
- manual_test_PASS
- manual_test_FAIL
- UAT_test_PASS
- UAT_test_FAIL
- Validation_PASS
- Validation_FAIL
- merge_to_master
variables:
start_pipeline:
stage: start_pipeline
script:
- if [[ -d "$USER_DIR" ]]; then echo -e "Direcory exists"; else sudo mkdir -p $USER_DIR; fi
- sudo chown -R root:gitlab-runner ${TARGET}/*
auto_testing:
stage: auto_testing
script:
- find . -type d -name "manifests" -exec chown -R gitlab-runner:gitlab-runner {} \;
- find . -type d -name "manifests" -exec puppet parser validate {} \;
- if [[ -d "$PRODUCTION_TARGET" ]]; then echo -e "Direcory exists"; else sudo mkdir -p $PRODUCTION_TARGET; fi
- if [[ -d "$LAB_TARGET" ]]; then echo -e "Direcory exists"; else sudo mkdir -p $LAB_TARGET; fi
manual_test_FAIL:
stage: manual_test_FAIL
script:
- echo "FAIL"
- exit 1;
when: manual
manual_test_PASS:
stage: manual_test_PASS
script:
- echo "PASS"
- sudo cp -r * ${TARGET}/${MODIFIED_COMMIT_USER}/
- sudo cp -r * ${LAB_TARGET}/
- sudo cp -r * ${PRODUCTION_TARGET}/
dependencies:
- auto_testing
I know this is likely too late to help you, but this is a known issue that they're targeting to fix in V9.0.
https://gitlab.com/gitlab-org/gitlab-ce/issues/26360

Resources