We've recently moved to Gitlab and have started using pipelines. We've set up a build server (an Ubuntu 16.04 instance) and installed a runner that uses a Shell executor but I'm unsure of how it actually executes the scripts defined in the .gitlab-ci.yml file. Consider the following snippet of code:
script:
- sh authenticate.sh $DEPLOY_KEY
- cd MAIN && sh deploy.sh && cd ..
- sh deploy_service.sh MATCHMAKING
- sh deauthenticate.sh
I was under the impression that it will just pipe these commands to Bash, and hence I was expecting the default Bash behaviour. What happens, however, is that the deploy.sh fails because of an ssh error ; Bash then continues to execute deploy_service.sh (which is expected behaviour) however this fails with a can't open deploy_service.sh error and the job terminates without Bash executing the last statement.
From what I understand, Bash will only abort on error if you do a set -e first and hence I was expecting all the statements to be executed. I've tried adding the set -e as the first statement but this makes no difference whatsoever - it doesn't terminate on the first ssh error.
I've added the exact output from Gitlab below:
Without set -e
$ cd MAIN && sh deploy.sh && cd ..
deploy.sh: 72: deploy.sh: Bad substitution
Building JS bundles locally...
> better-npm-run build
running better-npm-run in x
Executing script: build
to be executed: node ./bin/build
-> building js bundle...
-> minifying js bundle...
Uploading JS bundles to server temp folder...
COMMENCING RESTART. 5,4,3,2,1...
ssh: Could not resolve hostname $: Name or service not known
$ sh deploy_service.sh MATCHMAKING
sh: 0: Can't open deploy_service.sh
ERROR: Job failed: exit status 1
With set -e
$ set -e
$ cd MAIN && sh deploy.sh && cd ..
deploy.sh: 72: deploy.sh: Bad substitution
Building JS bundles locally...
> better-npm-run build
running better-npm-run in x
Executing script: build
to be executed: node ./bin/build
-> building js bundle...
-> minifying js bundle...
Uploading JS bundles to server temp folder...
COMMENCING RESTART. 5,4,3,2,1...
ssh: Could not resolve hostname $: Name or service not known
$ sh deploy_service.sh MATCHMAKING
sh: 0: Can't open deploy_service.sh
ERROR: Job failed: exit status 1
Why is it, without set -e, terminating on error (also, why is it terminating on the second error only and not the ssh error)? Any insights would be greatly appreciated.
Gitlab script block is actuality an array of shell scripts.
https://docs.gitlab.com/ee/ci/yaml/#script
Failure in each element of array will fail a whole array.
To workaround put your script block in some script.sh file
like
script:
- ./script.sh
I don't think your sh deploy.sh is generating a non-zero exit code.
You are using set -e to tell the current process to exit if a command exits with a non-zero return code, but you are creating a sub-process to run the shell script.
Here's a simple example script that I've called deploy.sh:
#!/bin/bash
echo "First."
echox "Error"
echo "Second"
If I run the script, you can see how the error is not handled:
$ sh deploy.sh
First.
deploy.sh: line 5: echox: command not found
Second
If I run set -e first, you will see it has no effect.
$ set -e
$ sh deploy.sh
First.
deploy.sh: line 5: echox: command not found
Second
Now, I add -e to the /bin/bash shebang:
#!/bin/bash -e
echo "First."
echox "Error"
echo "Second"
When I run the script with sh the -e still takes no effect.
$ sh ./deploy.sh
First.
./deploy.sh: line 3: echox: command not found
Second
When this script is run directly using bash, the -e takes effect.
$ ./deploy.sh
First.
./deploy.sh: line 3: echox: command not found
To fix your issue I believe you need to:
Add -e to the script shebang line (#!/bin/bash -e)
Call the script direct from bash using ./deploy.sh and not run the script through sh.
Bear in mind that if deploy.sh does fail then the cd .. will not run (&& means run the next command if the preceding one succeeded), which would mean you were in the wrong directory to run the deploy_service.sh. You would be better with cd MAIN; sh deploy.sh; cd .., but I suggest replacing your call to deploy.sh with simpler alternative:
script:
- sh authenticate.sh $DEPLOY_KEY
- (cd MAIN && sh deploy.sh)
- sh deploy_service.sh MATCHMAKING
- sh deauthenticate.sh
This is not wildly different, but will result in the cd MAIN && sh deploy.sh to be run in a sub-process (that's what the brackets do), which means that the current directory of the overall script is not affected. Think of it like "spawn a sub-process, and in the sub-process change directory and run that script", and when the sub-process finishes you end up where you started.
As other users have commented, you're actually running your scripts in sh, not bash, so all round this might be better:
script:
- ./authenticate.sh $DEPLOY_KEY
- (cd MAIN && ./deploy.sh)
- ./deploy_service.sh MATCHMAKING
- ./deauthenticate.sh
Related
I'm migrating shell scripts from solaris to redhat, using sh in solaris and bash in redhat.
Run cd /notexistsdir; echo 123 on both machines.
When the cd command reports an error in sh, it will not print 123, but in bash, it will print 123 after the cd command reports an error, indicating that the cd failure in sh will end the operation.
I tried to run cp /notexistsfile /aaa; echo 123 in sh again, although the cp command reported an error, it would print 123, and it did not end the operation.
i want to ask
What other commands in sh will end running after execution fails?
Why do sh and bash handle cd failure differently?
(using man cd to check the documentation, also found no information about the failure of the cd execution)
In bash, cd /notexistsdir && echo 123 should omit the 123 if cd produced an error.
Alternatively, you can call bash so that it fails on error. That would be done with bash -e.
Usually I make the following job in my .gitlab-ci.yml to execute commands on a remote server via SSH:
# The job
deploy:
script:
# I've omitted the SSH setup here
- |
ssh gitlab#example.com "
# Makes the server print the executed commands to stdout. Otherwise only the command output is printed. Required for monitoring and debug.
set -x &&
# Executes some commands
cd /var/www/example &&
command1 &&
command2 &&
command3 &&
command4 &&
command5
"
It works correctly, but the YAML code looks too complicated:
The set -x command is more a boilerplate than a useful code. It's not required for ordinary CI commands because GitLab CI prints them automatically.
&& on every line is boilerplate too. They make the execution stop when one of a commands fails. Otherwise the next commands will be executed when one fails (in contrast to ordinary job commands).
All the SSH commands are a single YAML string therefore editors don't highlight the comments and commands so the code is hard to read.
Is there a more clear and convenient way to execute multiple commands on a remote machine through SSH without the cons described above?
I'd like to not use external deployment tools like Ansible to keep the CD configuration as simple as possible (default POSIX/Linux commands are welcome). I'v also considered running each command in a separate ssh call but I'm afraid it may increase the job execution time because of multiple SSH connection establishments (but I'm not sure):
deploy:
script:
- ssh gitlab#example.com "cd /var/www/example"
- ssh gitlab#example.com "command1"
- ssh gitlab#example.com "command2"
# ...
A more concise and clear way is to use set -e. It makes the whole script fail when one of the commands fails. It lets you not use && on every line:
# The job
deploy:
script:
# I've omitted the SSH setup here
- |
ssh gitlab#example.com "
# Makes the server print the executed commands to stdout. Makes the execution stop when one of the commands fails.
set -x -e
# Executes some commands
cd /var/www/example
command1
command2
command3
command4
command5
# Even complex commands
if [ -f ./.env ]
then command6
else
echo 'Environment is not set up'
exit 1
fi
"
Keep your commands in a separate file remote.sh without set -x and &&:
#!/usr/bin/env bash
# Executes some commands
cd /var/www/example
command1
command2
command3
command4
command5
And use eval to run them on remote server:
deploy:
script:
- ssh gitlab#example.com "eval '$(cat ./remote.sh)'"
This approach will keep YAML simple and clean and meet all your requirements.
I have a crontab command set up like below
*/2 * * * * cd dir && dir/keepalive.ksh qa >> /var/log/test/keepalive.log
the keepalive.sh script essentially calls a start script with 'test' argument which executes the following script
print "Start the following proccesses"
if [ ${ENV} == "qa" ]
then
dir="my path"
print "Start QA Server"
pm2 start ${dir}/server.js -- app-env=qa db-env=qa
fi
exit
Problem: The pm2 command never starts up the process. I also tried to use the full path of pm2 (/usr/local/bin/pm2) but still no luck. I can see the output of the print statements which means the script is getting executed as expected.
Any idea what might be going wrong?
I ran into the similar problem before and it turned out I need to prepend the pm2 command with the exact path to my node executable like this
/usr/local/bin/node /usr/local/bin/pm2 start myapp.js
So in your case, you can try
/path/to/node /usr/local/bin/pm2 start ${dir}/server.js -- app-env=qa db-env=qa
Where /path/to/node needs to be replaced with the output of which node in your terminal.
I have a script that needs to check if a lockfile exists that only root can access and then the script runs the actual test script that needs to be run as a different user.
The problem is the test script is supposed to generate xml files and those files do not exist. (aka I can't find them)
Relevant part of the script
if (mkdir ${lockdir} ) 2> /dev/null; then
echo $$ > $pidfile
trap 'rm -rf "$lockdir"; exit $?' INT TERM EXIT
if [ -f "$puppetlock" ]
then
su -c "/opt/qa-scripts/start-qa-test.sh > /var/log/qaTests/test-$(date +\"m-%d-%T\".log" -m "qaUser"
lockdir is what gets created when the test is run to signify that the test process has begun.
puppetlock checks if puppet is running by looking for the lock file puppet creates.
qaUser does not have the rights to check if puppetlock exists.
start-qa-test.sh ends up calling java to execute an automated test. My test-date.log file displays what console would see if the test was run.
However the test is supposed to produce some xml files in a directory called target. Those files are missing.
In case it's relevant start-qa-test.sh is trying to run something like this
nohup=true
/usr/bin/java -cp .:/folderStuff/$jarFile:/opt/folderResources org.junit.runnt.JUNITCore org.some.other.stuff.Here
Running start-qa-test.sh produces the xml output in the target folder. But running it through su -c it does not.
Edit
I figured out the answer to this issue.
I changed the line to
su - qaUser -c "/opt/qa-scripts/start-qa-test.sh > /var/log/qaTests/test-$(date +\"m-%d-%T\".log"
That allowed the output to show up at the /home/qaUser
Try redirecting the output of stout and stderr in the line:
su -c "/opt/qa-scripts/start-qa-test.sh > /var/log/qaTests/test-$(date +\"m-%d-%T\".log" -m "qaUser" 2>&1
(I'm new in shell script.)
I've been stuck with this issue for a while. I've tried different methods but without luck.
Description:
When my script attempt to run another script (SiebelMessageCreator.sh, which I don't own) it prompts:
-bash: ./SiebelMessageCreator.sh: No such file or directory
But the file exists and has execute permissions:
-rwxr-xr-x 1 owner ownergrp 322 Jun 11 2015 SiebelMessageCreator.sh
The code that is performing the script execution is:
(cd $ScriptPath; su -c './SiebelMessageCreator.sh' - owner; su -c 'nohup sh SiebelMessageSender.sh &' - owner;)
It's within a subshell because I first thought that it was throwing that message because my script was running in my home directory (When I run the script I'm root and I've moved to my non-root home directory to run the script because I can't move my script [ policies ] to the directory where the other script resides).
I've also tried with the sh SCRIPT.sh ./SCRIPT.sh. And changing the shebang from bash to ksh because the SiebelMessageCreator.sh has that shell.
The su -c 'sh SCRIPT.sh' - owner is necessary. If the script runs as root and not as owner it brokes something (?) (that's what my partners told me from their experience executing it as root). So I execute it as the owner.
Another thing that I've found in my research is that It can throw that message if it's a Symbolic link. I'm really not sure if the content of the script it's a symbolic link. Here it is:
#!/bin/ksh
BASEDIRROOT=/path/to/file/cpp-plwsutil-c
ore-runtime.jar (path changed on purpose for this question)
java -classpath $BASEDIRROOT com.hp.cpp.plwsutil.SiebelMessageCreator
exitCode=$?
echo "`date -u '+%Y-%m-%d %H:%M:%S %Z'` - Script execution finished with exit code $exitCode."
exit $exitCode
As you can see it's a very siple script that just call a .jar. But also I can't add it to my script [ policies ].
If I run the ./SiebelMessageCreator.sh manually it works just fine. But not with my script. I suppose that discards the x64 x32 bits issue that I've also found when I googled?
By the way, I'm automating some tasks, the ./SiebelMessageCreator.sh and nohup sh SiebelMessageSender.sh & are just the last steps.
Any ideas :( ?
did you try ?
. ./SiebelMessageCreator.sh
you can also perform which sh or which ksh, then modify the first line #!/bin/ksh