If condition is failing in the GitLab CI - linux

following is my Gitlab CI code:-
stages:
- check
variables:
JIRA_HEADER: "Accept: application/json"
jira:
stage: check
before_script:
#- apk add jq curl
- apk add --no-cache bash jq curl
image: python:3.7.4-alpine3.9
script:
- export MERGE_REQUEST_JIRA_ID=$(echo ${CI_MERGE_REQUEST_TITLE} | sed -r "s/^([A-Za-z][A-Za-z0-9]+-[0-9]+).*/\1/")
- echo $CI_MERGE_REQUEST_TITLE
- export JIRA_DETAIL=$(curl -u ${JIRA_USERNAME}:${JIRA_PASSWORD} -H "${JIRA_HEADER}" -X GET https://${JIRA_SERVER}/rest/api/2/issue/${MERGE_REQUEST_JIRA_ID}?fields=status)
- echo $JIRA_DETAIL
# extract the JIRA key id, this also validates JIRA issue referenced is valid
- export JIRA_KEY_ID=$(echo ${JIRA_DETAIL} | jq -e '.key')
- echo $JIRA_KEY_ID
# extract the JIRA status
- export JIRA_STATUS=$(echo ${JIRA_DETAIL} | jq '.fields.status.name')
- echo $JIRA_STATUS
- |
if [[ "$JIRA_STATUS" == "^(Done|Completed|Closed)$" ]]
then
echo "Invalid JIRA (Done/Completed/Closed) found!"
exit 1
else echo "Valid JIRA Id found!"
fi
only:
- merge_requests
I'm trying to validate the JIRA status by calling its API after retrieving Jira id from title of Merge Request. There is a problem in the If condition below if [[ "$JIRA_STATUS" == "^(Done|Completed|Closed)$" ]] as it is not validating it properly. Every time, the else condition is getting executed and printing the message as Valid JIRA Id found!
I would really appreciate if someone can help me to fix this minor issue. I want to gracefully exit the job with this message in the if block as Invalid JIRA (Done/Completed/Closed) found! whenever the Jira status found to be in any of the given values as Done, Completed or Closed.

I'm finally able to resolve this issue by modifying the code like below:-
- |
if test -z "$(echo ${JIRA_STATUS} | sed -r "s/\"(Done|Completed|Closed)\"//")"
then
echo "Not a valid Jira (Done/Completed/Closed)"; exit 1
else
echo "Valid Jira found!"; echo $?
fi
I had used the test command along with if-else condition in Linux to make it work

Related

Gitlab Yamls does not works properly Job needs

I'm facing some issues and would be glad if someone can help me.
My main goal here it's being able to save at the same pipeline, more than one job. The error it’s because the jobs does not exist for the specific branch, since I have 6 branches I don't want to modified one by one. I would like to have only one Yaml version.
When I've tried to save, GitLab show me the following errors.
Found errors in your .gitlab-ci.yml:
jobs:deploy_dc_manual:needs:need job should be a string
You can test your .gitlab-ci.yml in CI Lint.
This, it is a piece of the code. Example if I am in the branch feature the job needs it´s "feature_package_build" If I am at integration branch will be expected "int_package_build".
feature_package_build:
extends: .build
only:
- /^feature\/.*/
script:
# GitLab API query
- LAST=$(curl -s "sensitive data" | jq '.[0] | .sha' | sed '1q;d' | sed 's:^.\(.*\).$:\1:')
- >
if [ "$OLDER_COMMIT" == "none" ]; then
node_modules/sfdx-cli/bin/run sfpowerkit:project:diff -d package -r ${LAST} -x --loglevel debug
elif [ "$OLDER_COMMIT" != "none" ]; then
node_modules/sfdx-cli/bin/run sfpowerkit:project:diff -d package -r $OLDER_COMMIT -x --loglevel debug
fi
int_package_build:
extends: .build
only:
- integration
script:
# GitLab API query
- LAST=$(curl -s "sensitive data" | jq '.[0] | .sha' | sed '1q;d' | sed 's:^.\(.*\).$:\1:')
- >
if [ "$OLDER_COMMIT" == "none" ]; then
node_modules/sfdx-cli/bin/run sfpowerkit:project:diff -d package -r ${LAST} -x --loglevel debug
elif [ "$OLDER_COMMIT" != "none" ]; then
node_modules/sfdx-cli/bin/run sfpowerkit:project:diff -d package -r $OLDER_COMMIT -x --loglevel debug
fi
uat_package_build:
extends: .build
only:
- uat
script:
# GitLab API query
- LAST=$(curl -s "sensitive data" | jq '.[0] | .sha' | sed '1q;d' | sed 's:^.\(.*\).$:\1:')
deploy_DC_Manual:
extends:
- .deployDC_Manual
needs:
- job:
if [uat_package_build]; then
fi
if [feature_package_build]; then
fi
only:
- /^feature\/.*/
- integration
- uat
- release
- master
script:
- nomDeployedDC=${NONDEPLOYEDDC}
- >
if [ -f package/destructiveChanges.xml ] && [ "$VALIDATE" == "no" ]; then
I assume what you are looking for is the optional-flag for the needs keyword like
deploy_DC_Manual:
extends:
- .deployDC_Manual
needs:
- job: uat_package_build
optional: true
- job: feature_package_build
optional: true
As this makes the needs conditional - but please be aware, that this also means if you mess up your rules, and none of those two is available, your job might be still executed.

Gitlab CI: Facing error "Syntax error near unexpected token `else' " in gitlab ci job

Here is my yml file:
stages:
- stage1
job1:
stage: stage1
script:
- >
#!/bin/sh
cd .
echo "checking whether the ctl file exists"
if [ -s something.ctl ]
then
echo "exists"
else
echo "does not exist, exiting gitlab pipeline"
exit -1
fi
When the pipeline runs it fails with exit code 2 and I see this message in the log:
Syntax error near unexpected token `else'
What do I need to change here?
This script section work (using bash shell) :
script:
- cd .
- echo "checking whether the ctl file exists"
- >
if [ -s something.ctl ]; then
echo "exists";
else
echo "does not exist, exiting gitlab pipeline";
exit 1;
fi
Also be aware that to exit a job properly, code must be 0 (succeed) or 1 (fail)
The issue resolved after I replaced "- >" with "- |-"
so now my file looks like
script:
- |-
echo "..."
echo....

Check if task exist - AWS ECS

How do you check from cli or using a shell script if the task exists?
The documentation is confusing. Is using the --task-definition option as family.
The command
TASK=$(aws ecs describe-task-definition --task-definition ${TASK_NAME}
it errors saying
An error occurred (ClientException) when calling the DescribeTaskDefinition operation: Unable to describe task definition.
Any ideas?
Does this help?
#!/bin/bash
ecs_arn=$1
aws_result=$(aws ecs describe-task-definition --task-definition "$ecs_arn" --query 'taskDefinition.[taskDefinitionArn,status]' --output json 2>/dev/null)
# is it present?
if [ $(echo "$aws_result" | grep -i "$ecs_arn" | wc -l) -ne 1 ]; then
echo "task-definition $ecs_arn doesn't exist."
elif [ $(echo "$aws_result" | grep -i active | wc -l) -ne 1 ]; then
echo "task-definition $ecs_arn exist but not active."
else
echo "task-definition $ecs_arn exist and it's active."
fi
# end
Save it as bash script and execute it like:
./script.sh "arn:aws:ecs:REGION-HERE:ACCOUNT-NAME:............."
You can extract all definition ARNs with:
aws ecs list-task-definitions
I'm redirecting std errors to /dev/null because the api returns error like yours when the task doesn't exist.
You can modify the validations as you like in your case.
Documentation used: https://docs.aws.amazon.com/cli/latest/reference/ecs/index.html

Curl Command in Shell Script in Azure Pipeline not working

I'm trying to run a script which provides the status code of some url, using azure pipeline.
My Azure Yaml file:
pool:
vmImage: ubuntu-16.04
steps:
- script: echo Hello
displayName: ' Welcome'
- script: cat webpages.txt
displayName: 'display file'
- script: curl -s -w '%{http_code}\n' -o /dev/null https://www.google.co.in
displayName: 'Checking Curl Code'
- script: cat -v script.sh
displayName: 'Cariage retrun'
- task: ShellScript#2
inputs:
scriptPath: script.sh
My Script.sh file
#!/bin/sh
while read line ; do echo "$line - `curl -s -w '%{http_code}\n' -o /dev/null $line`" ;done < webpages.txt
webpages.txt file
https://www.vodafone.co.uk/good-stuff
https://www.vodafone.co.uk/help-and-information/cancel-your-account
https://www.vodafone.co.uk/help-and-information/complaints
https://www.vodafone.co.uk/help-and-information/complaints/code-of-practice
https://www.vodafone.co.uk/help-and-information/costs-and-charges
https://www.vodafone.co.uk/help-and-information/costs-and-charges/call-and-text-charges
https://www.vodafone.co.uk/help-and-information/costs-and-charges/data-charges
Problem
When I run my pipeline, the curl command is not working
and output comes as
2020-11-17T09:59:05.4094324Z ##[section]Starting: ShellScript
2020-11-17T09:59:05.4102534Z ==============================================================================
2020-11-17T09:59:05.4102904Z Task : Shell script
2020-11-17T09:59:05.4103204Z Description : Run a shell script using Bash
2020-11-17T09:59:05.4103453Z Version : 2.165.2
2020-11-17T09:59:05.4103701Z Author : Microsoft Corporation
2020-11-17T09:59:05.4104086Z Help : https://learn.microsoft.com/azure/devops/pipelines/tasks/utility/shell-script
2020-11-17T09:59:05.4104506Z ==============================================================================
2020-11-17T09:59:05.7637654Z [command]/bin/bash /home/vsts/work/1/s/script.sh
2020-11-17T09:59:05.7692023Z https://www.vodafone.co.uk/good-stuff
2020-11-17T09:59:05.7700780Z - 000
2020-11-17T09:59:05.7797276Z https://www.vodafone.co.uk/help-and-information/cancel-your-account
2020-11-17T09:59:05.7798378Z - 000
2020-11-17T09:59:05.7851183Z https://www.vodafone.co.uk/help-and-information/complaints
2020-11-17T09:59:05.7866944Z - 000
2020-11-17T09:59:05.7908420Z https://www.vodafone.co.uk/help-and-information/complaints/code-of-practice
2020-11-17T09:59:05.7909144Z - 000
2020-11-17T09:59:05.7967261Z https://www.vodafone.co.uk/help-and-information/costs-and-charges
2020-11-17T09:59:05.7967920Z - 000
2020-11-17T09:59:05.8023329Z https://www.vodafone.co.uk/help-and-information/costs-and-charges/call-and-text-charges
2020-11-17T09:59:05.8024443Z - 000
2020-11-17T09:59:05.8095527Z https://www.vodafone.co.uk/help-and-information/costs-and-charges/data-charges
but if I replace my curl with
curl -s -w '%{http_code}\n' -o /dev/null https://www.vodafone.co.uk/good-stuff
it gives the output at 200.
When I'm running this locally it works perfectly:
$ while read line ; do echo "$line - `curl -s -w '%{http_code}\n' -o /dev/null $line`" ;done < webpages.txt
https://www.vodafone.co.uk/good-stuff - 200
https://www.vodafone.co.uk/help-and-information/cancel-your-account - 200
(...)
I notice that a \n character is printed in your echo $line, that's probably causing the issue. What could solve it, is:
replacing $line by ${line},
replacing echo by echo -n to omit newlines.
So this is what I did,
I replaced the whole curl command with a simple curl $line
It gave me an error : curl: (3) Illegal characters found in URL
So, I figured my URL is adding some unrequired values.
I replaced my do with
line=${line%$'\r'}
while read line ; do line=${line%$'\r'} ; echo "$line - `curl -s -w '%{http_code}\n' -o /dev/null $line`"; done < webpages.txt
and Voila it works!

GitLab CI syntax to write FOR loop statement?

Below is the script mentioned in the gitlab-ci.yml file. This GitLab CI configuration is valid. But, when the CI/CD build is run, the job fails. Is it something to do with the FOR loop syntax?
deploy_dv:
stage: deploy_dv
variables:
GIT_STRATEGY: none
script:
- echo "Deploying Artifacts..."
- echo "Configure JFrog CLI with parameters of your Artifactory instance"
- 'c:\build-tools\JFROG-CLI\jfrog rt config --url %ARTIFACTORY_WEBSITE% --user %ARTIFACTORY_USER% --apikey %APIKEY%'
- 'cd ..\artifacts'
- 'SETLOCAL ENABLEDELAYEDEXPANSION'
- FOR %%i in (*) do (
'c:\build-tools\curl\bin\curl.exe --header "PRIVATE-TOKEN:%HCA_ACCESS_TOKEN%" --insecure https://code.example.com/api/repository/tags/%CI_COMMIT_TAG% | c:\build-tools\jq\jq-win64.exe ".release.description" > temp.txt'
'set /p releasenote=<temp.txt'
'rem del temp.txt'
'set mydate=%DATE:~6,4%-%DATE:~3,2%-%DATE:~0,2%'
'c:\build-tools\JFROG-CLI\jfrog rt u "%%i" %ARTIFACTORY_ROOT_PATH%/%PROJECT_NAME%/%%i --build-name=%%i --build-number=%BUILDVERSION% --props releasenote=%releasenote%;releaseversion=%BUILDVERSION%;releasedate=%mydate% --flat=false'
)
- '%CURL% -X POST -F token=%REPOSITORY_TOKEN% -F ref=master -F "variables[RELEASE]=false" -F "variables[PROGRAM]=test" --insecure https://code.example.com/api/repository/trigger'
only:
- /^(dv-)(\d+\.)(\d+\.)(\d+)$/
I get this below error:
$ echo "Deploying Artifacts..."
"Deploying Artifacts..."
$ echo "Configure JFrog CLI with parameters of your Artifactory instance"
"Configure JFrog CLI with parameters of your Artifactory instance"
$ c:\build-tools\JFROG-CLI\jfrog rt config --url %ARTIFACTORY_WEBSITE% --user %ARTIFACTORY_USER% --apikey %APIKEY%
Artifactory server ID [Default-Server]: $ cd ..\artifacts
$ SETLOCAL ENABLEDELAYEDEXPANSION
$ FOR %%i in (*) do ( 'c:\build-tools\curl\bin\curl.exe --header "PRIVATE-TOKEN:%HCA_ACCESS_TOKEN%" --insecure https://code.example.com/api/repository/tags/%CI_COMMIT_TAG% | c:\build-tools\jq\jq-win64.exe ".release.description" > temp.txt' 'set /p releasenote=<temp.txt' 'rem del temp.txt' 'set mydate=%DATE:~6,4%-%DATE:~3,2%-%DATE:~0,2%' 'c:\build-tools\JFROG-CLI\jfrog rt u "%%i" %ARTIFACTORY_ROOT_PATH%/%PROJECT_NAME%/%%i --build-name=%%i --build-number=%BUILDVERSION% --props releasenote=%releasenote%;releaseversion=%BUILDVERSION%;releasedate=%mydate% --flat=false' )
The filename, directory name, or volume label syntax is incorrect.
ERROR: Job failed: exit status 255
Since there is still no good answer to this question, I will give it a try. I used this snippet to start multiple Docker builds for every directory in my repository. Notice the |+ and the > characters, which lets you put multi-line commands in YAML and are part of GitLab syntax.
Linux example:
build:
stage: loop
script:
- |+
for i in $(seq 1 3)
do
echo "Hello $i"
done
Windows example:
build:
stage: loop
script:
- >
setlocal enabledelayedexpansion
for %%a in ("C:\Test\*.txt") do (
set FileName=%%~a
echo Filename is: !FileName!
)
endlocal
Here is a working example of a job in a .gitlab-ci with a loop running on GNU/Linux OS and using Sh/Bash shell :
edit:
stage: edit
script:
- for file in $(find ${CI_PROJECT_DIR} -type f -name deployment.yml)
do
CURRENT_IMAGE=$(grep "image:" $file | cut -d':' -f2- | tr -d '[:space:]' | cut -d':' -f3)
sed -ie "s/$CURRENT_IMAGE/$VERSION/g" "$file"
done
only:
- master
I'm not an expert on Gitlab-Runner on Windows but Windows Batch is default shell used but you can also use Powershell.
In .gitlab.yml anything you write under "script" is shell. Thus for loop will be same as it works in shell script.
for var in ${NAME_1} ${NAME_2} ${NAME_3} ; do
*----computations----*
done

Resources